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Many  of  today's  military  systems  cannot  be 
adequately  operated,  maintained,  or  supported 
because  the  Department  of  Defense  does  not 
pay  enough  attention  to  logistic  support,  hu¬ 
man  factors,  and  quality  assurance  during  the 
design  phase  of  the  acquisition  process.  These 
problems  deter  the  systems'  effectiveness  to 
defend  our  country  in  case  of  war. 

G  AO.therefore.makes  recommendations  to  im¬ 
prove  the  man^ement  and  planning  of  owner¬ 
ship  considerations  that  have  an  impact  on 
the  effectiveness  of  a  weapon  system. 


DTIC 

Sei-ecte|% 

APR  2  6  19821  ■ 


DISTBtBUnON  STATEMENT  A 

Appioved  fos  public  ieleaa«( 
Distribution  Unlimited 


«ee/psAD-8i-i7 
JANUARY  29. 1M1 


82  04  27  163 


COMPTROLLER  GENERAL  OP  THE  UNITED  STATES 
WASHINOTON.  D.C.  MM* 


B-201505 


To  the  President  of  the  Senate  and  the 
Speaker  of  the  House  of  Representatives 

This  report  discusses  weaknesses  in  the  Department  of 
Defense's  major  system  acquisition  process  which  are  contri¬ 
buting  to  the  operation,  maintenance,  and  support  problems 
of  many  deployed  weapon  systems.  We  consider  the  message 
of  this  report  to  be  particularly  important  considering 
the  problems  our  Armed  Forces  are  having  with  weapon  systems . 


This  review  was  undertaken  in  an  effort  to  identify 
changes  which  are  needed  in  the  acquisition  process  to  mini¬ 
mize  problems  with  future  weapon  systems. 


Copies  of  this  report  are  being  sent  to  the  Director, 
Office  of  Management  and  Budget,  and  to  the  Secretaries 
of  Defense,  Army,  Navy,  and  Air  Force. 
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COMPTROLLER  GENERAL'S 
REPORT  TO  THE  CONGRESS 


EFFECTIVENESS  OF  U.S.  FORCES 
CAN  BE  INCREASED  THROUGH 
IMPROVED  WEAPON  SYSTEM  DESIGN 


DIGEST 

The  United  States'  ability  to  fight  a  war 
may  be  severely  hampered  because  many  of 
the  aircraft,  ships,  tanks,  ordnance,  and 
other  systems  the  Armed  Forces  must  use  are 
suffering  from  numerous  problems.  While 
these  systems  may  have  the  capability  to  per¬ 
form  their  missions,  it  is  often  of  little 
value  because  not  all  the  systems  can  be 
adequately  operated,  maintained,  or  sup¬ 
ported  . 

GAO  has  reported  on  these  problems  in  the 
past  and  believes  that  many  of  them  can 
be  traced  to  the  Department  of  Defense ' s 
(DOD's)  system  acquisition  process,  oarticu- 
larly  the  early  phases  before  system  design 
is  set.  The  pressures  to  attain  specific 
performance  goals,  such  as  speed,  range, 
and  firepower,  within  tight  time  and  cost 
constraints  have  often  led  management  to 
trade-off  or  otherwise  not  give  adequate 
attention  to  long  term  "ownership  considera¬ 
tions  . " 

GAO  examined  three  ownership  considerations 
which  appear  to  be  among  the  most  prominent 
detractors  from  the  effectiveness  of  deployed 
systems — logistic  support,  human  factors, 
and  quality  assurance.  DOD  has  made  changes 
to  direct  more  attention  to  these  three 
factors,  but  GAO  believes  that  several  addi¬ 
tional  actions  need  to  be  taken  if  these 
considerations  are  to  receive  adequate  empha¬ 
sis  in  the  design  of  military  systems  and 
equipment.  These  actions  will  have  to  include 
greater  support  by  the  Congress  and  DOD  for 
activities,  such  as  logistics  and  human  fac¬ 
tors  research,  testing,  and  analysis. 


Tw  SNet  Upon  rcmovil,  the  report 
cover  date  should  be  noted  hereon. 
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LOGISTIC  SUPPORT 


Logistic  support  requirements,  such  as  replace¬ 
ment  parts,  tools,  test  equipment,  technical 
manuals  and  facilities,  and  the  time  and  cost 
of  providing  these,  are  not  being  adequately 
addressed  when  designing  systems  and  equip¬ 
ment.  This  contributes  to  low  system  readi¬ 
ness  rates  when  deployed  because  the  military 
supply  and  maintenance  systems  cannot  meet 
the  demands  placed  on  them. 

DOD  is  now  placing  greater  emphasis  on  logis¬ 
tic  support  requirements  during  the  acquisi¬ 
tion  process,  including  adopting  a  policy 
that  supportability  is  as  important  as  cost, 
schedule,  and  performance.  The  positive  impact 
these  changes  need  to  have  may  never  fully 
materialize  because: 

— The  process  for  interfacing  logistic  consid¬ 
erations  with  other  design  considerations— 
logistic  support  analysis — is  very  difficult 
to  do.  Much  of  the  data  needed  for  design 
decisions  is  difficult  to  obtain;  there 
is  a  shortage  of  trained  people  to  do  the 
analysis;  and  the  analysis  can  be  costly, 
especially  if  duplicative  analyses  are  not 
eliminated.  (See  pp.  22  and  23.) 

— The  quantitative  analysis  needed  to  assess 
logistic  plans,  resources,  and  support- 
related  parameters  for  meeting  system  readi¬ 
ness  goals  may  be  very  difficult  to  perform 
because  the  analytical  models  for  making 
such  assessments  may  not  be  adequate.  (See 
p.  23.) 

— There  is  very  little  guidance  to  ensure  that 
critical  program  documents  contain  the  lan¬ 
guage  needed  to  get  systems  designed  that 
are  supportable.  (See  p.  24.) 

— Testing  a  system's  supportability  before 
it  is  deployed  is  difficult  to  do,  requires 
dedicated  test  time  and  articles,  and  is 
expensive.  (See  pp.  24  and  25.) 

— There  is  little  incentive  for  management  to 
either  invest  development  funds  or  to 
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trade-off  technical  performance  to  improve 
the  supportability  of  a  system  because  it 
is  very  difficult  to  quantify  the  benefits 
of  such  investments  and  trade-offs.  (See 
p.  25.) 

HUMAN  FACTORS 

Limitations  such  as  skill  levels,  proficiency, 
availability,  environmental  stress,  and  fa¬ 
tigue  of  the  personnel  who  operate  and  maintain 
military  systems  contribute  to  human-induced 
system  failures.  Indications  are  that  these 
types  of  failures  are  quite  high.  New  policy 
emphasis  on  human  limitations  in  the  design 
of  systems  may  have  a  very  limited  impact 
because: 

— Human  factor  specifications,  standards,  and 
handbooks  used  in  designing  and  developing 
systems  and  equipment  do  not  adequately 
address  human  limitations.  (See  p.  31.) 

—There  are  no  common  methodologies  and  data 
sources  for  use  by  system  designers  in 
forecasting  skill  levels  of  future  military 
personnel.  (See  pp.  31  and  32.) 

— DOD  testing  policies  and  procedures  do  not 
tend  to  identify  and  resolve  potential 
human-induced  failures  during  the  develop¬ 
mental  stages  of  the  acquisition  process. 

(See  pp.  32  and  33.) 

QUALITY  ASSURANCE 


Systems  being  deployed  are  not  as  reliable  as 
they  are  intended  to  be.  Part  of  the  problem 
is  that  the  reliability  inherent  in  system 
designs  is  being  lost  in  the  transition  from 
design  to  production  and  deployment. 

The  extent  of  the  problem  is  very  difficult 
to  quantify,  but  its  existence  points  up  the 
need  for  designing  systems  which  can  be  manu¬ 
factured  to  the  tolerances  called  for  in  the 
specifications  and  then  tested  to  confirm  com¬ 
pliance.  DOD  recognizes  the  problem,  but 
attempts  to  place  greater  emphasis  on  design¬ 
ing  these  types  of  features  into  systems  are 
hampered  because: 
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— There  ia  a  lack  of  guidance  to  project 
managers  on  how  to  evaluate  designs  for 
quality  assurance.  Part  of  the  problem  may 
be  that  there  is  only  one  person  in  the 
Office  of  the  Secretary  of  Defense  assigned 
to  the  policy  aspects  of  quality  assurance. 
(See  p.  38. ) 

— Files  on  contractor  quality  histories  have 
not  been  fully  established  and  the  cross¬ 
service  product  quality  deficiency  reporting 
requirement  has  not  been  fully  implemented. 
(See  p.  38 . ) 

— Government  engineers  with  adequate  training 
in  quality  assurance  are  in  short  supply. 
(See  p.  39.) 

RECOMMENDATION  TO  THE  CONGRESS 


GAO  recommends  that  the  Congress  direct  more 
attention  during  its  deliberations  on  DOD's 
budget  to  such  matters  as  logistic  support, 
human  factors,  and  quality  assurance  considera¬ 
tions  in  the  design  and  development  of  weapon 
systems . 

RECOMMENDATIONS  TO  THE 
SECRETARY  OF  DEFENSE 

Logistic  support 

GAO  recommends  that  the  Secretary  of  Defense 

— develop  new  or  modify  current  data  report¬ 
ing  procedures  to  provide  the  information 
needed  for  performing  logistic  support 
analyses, 

— establish  logistic  support  research  and 
study  programs  to  develop  improved  quanti¬ 
tative  assessment  methods  (see  p.  26), 

— provide  detailed  guidance  to  ensure  that 
critical  program  documents  contain  the. 
language  needed  to  obtain  systems  which 
are  supportable,  and 
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— provide  for  improved  testing  and  evaluation 
of  the  supportability  of  systems  before  they 
are  deployed. 

Human  factors 


GAO  recommends  that  the  Secretary  -of  Defense 

— modify  human  factor  specifications,  stand¬ 
ards,  and  handbooks  used  in  system  design 
so  that  they  adequately  address  all  human 
limitations  which  can  result  in  human- 
induced  system  failures, 

— develop  common  methodologies  and  data 
sources  for  use  by  system  designers  in 
forecasting  skill  levels  of  military  per¬ 
sonnel  5  to  10  years  in  the  future,  and 

— ensure  that  all  major  systems  are  subjected 
to  adequate  testing  and  examination  from  a 
human  factors  standpoint  throughout  the 
acquisition  process,  particularly  in  the 
developmental  stages. 

Quality  assurance 

GAO  recommends  that  the  Secretary  of  Defense 

--produce  comprehensive  guidance  for  program 
managers  as  to  how  a  design  should  be 
evaluated  for  quality  assurance, 

— ensure  that  the  quality  history  files  on 
contractors  are  fully  established  and 
that  the  cross-service  quality  deficiency 
reporting  requirement  is  fully  implemented, 
and 

— strengthen  the  quality  assurance  work  force 
so  as  to  permit  their  active  involvement 
in  the  design  phase  of  the  acquisition  process. 

AGENCY  COMMENTS 

DOD  agreed  with  GAO's  findings,  conclusions, 

and  recommendations.  (See  app.  111.)  Changes 

were  made  to  the  report,  where  appropriate, 

to  reflect  specific  comments. 
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CHAPTER  1 


INTRODUCTION 

Each  year  the  Department  of  Defense  (DOD)  spends  billions 
of  dollars  to  research,  design,  develop,  produce,  and  support 
major  weapon  systems.  The  process  for  acquiring  these  systems 
is  extremely  complicated  and  influenced  by  a  host  of  factors 
(in  addition  to  technical  performance  and  acquisition  cost 
and  schedule)  that  affect  the  effectiveness  of  a  weapon  sys¬ 
tem.  These  include; 

Survivability  Supportability 

Vulnerability  Compatibility 

Operational  suitability  Reliability 

Transportability  Maintainability 

Availability  Durability 

Interoperability  Quality 

These  factors  called  "ownership  considerations"  must  be 
balanced  during  the  acquisition  process  by  the  program  manager 
in  an  endless  juggling  act  to  acquire  a  system  which  meets 
mission  requirements  within  program  acquisition  cost  and 
schedule  constraints. 

Many  of  the  problems  being  experienced  with  the  opera¬ 
tion,  maintenance,  and  support  of  weapon  systems  can  be 
related  back  to  the  extent  to  which  ownership  considerations 
were  a  factor  in  the  early  acquisition  process.  This  report 
focuses  on  the  things  that  must  be  done  to  give  some  of 
these  considerations  more  weight  in  the  acquisition  process 
balancing  act. 

Within  the  past  few  years  DOD  has  revised  policies, 
issued  guidance,  and  made  organizational  changes  designed 
to  elevate  ownership  considerations  to  a  higher  level  of 
attention  throughout  the  acquisition  process.  Some  of  DOD's 
actions  are  very  recent,  and  their  impact  remains  to  be  seen. 

OBJECTIVES,  SCOPE,  AND  METHODOLOGY 

Because  of  the  concern  expressed  by  many  military  and 
other  observers  over  the  combat  readiness  of  U.S.  Armed 
Forces,  we  initiated  this  review  to  (1)  identify  some  of  the 
more  prominent  causes  of  problems  with  acquiring  and  fielding 
major  weapon  systems  and  (2)  recommend  some  meaningful  actions 
to  reduce  problems  with  deployed  systems  in  the  future.  We 
were  specifically  concerned  with  identifying  weaknesses 
in  the  acquisition  process  which  were  contributing  to  the 
deployment  of  weapon  systems  which  are  difficult  to  operate. 


1 


maintain,  and  support.  The  review  was  not  intended  to  pro¬ 
vide  an  indepth  examination  of  the  entire  acquisition 
process,  but  rather  to  focus  on  those  areas  of  the  process 
which  we  believe  offer  significant  opportunity  for  improve¬ 
ment  . 


This  review  included  visits  to  many  locations  in  the 
United  States  and  Europe.  (See  app.  I  for  the  list  of  loca¬ 
tions  visited.) 

We  approached  this  job  as  follows; 

— Through  research  and  visits  to  users,  weapon  systems 
were  selected  for  review  which  exhibited  problems  that 
we  believed; 

1.  Were  mission  significant. 

2 .  Might  have  been  prevented  or  anticipated  during 
the  acquisition  cycle. 

3.  Were  being  experienced  by  more  than  one  piece  of 
equipment  of  the  same  type. 

4.  Could  be  researched  because  information  would  prob 
ably  still  be  available  on  acquisition  process 
decisions . 

— These  problems  were  traced  back  into  the  acquisition 
cycle  to  determine  root  causes  and  the  trade-off  deci¬ 
sion  which  led  to  these  causes. 

— We  were  able  to  determine  most  problem  causes.  All 
of  the  causes  fell  into  several  broad  categories,  and 
all  seemed  to  involve  inadequate  attention  to  owner¬ 
ship  consideration  in  weapon  system  design.  Three 
major  early  design  phase  considerations  were  selected 
for  further  work . 

— A  detailed  review  into  the  areas  of  logistic  support, 
human  factors,  and  quality  assurance  was  conducted 
with  emphasis  placed  on  how  these  ownership  considera¬ 
tions  are  addressed  in  the  early  phases  of  the  acquisi 
tion  cycle  (mainly  between  milestones  0  and  I) .  Much 
of  the  information  was  obtained  through  discussions 
with  DOD  and  industry  representatives  involved  with 
the  acquisition  of  weapon  systems. 

— Improvements  to  the  acquisition  cycle  were  postulated 
to  hopefully  alleviate  similar  problems  in  the  future. 
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Our  office  has  issued  numerous  reports  that  relate  to 
readiness,  logistic  support,  military  personnel,  the  DOD 
acquisition  process,  and  other  matters  discussed  in  this 
report.  For  a  listing  of  some  of  the  most  relevant  reports 
issued  since  January  1979,  see  appendix  II.  A  February  1979 
DOD  study,  "Defense  Resource  Management  Study,"  also  reported 
on  some  of  the  issues  discussed  in  this  report.  The  study 
report  recommended  increased  emphasis  on  ownership  considera¬ 
tions  in  the  acquisition  process  and  identification  and 
application  of  innovative  support  concepts  for  new  systems 
being  developed . 

COMMEMTS  OF  OUR  PROCUREMENT 
ADVISORY  PANEL 

A  draft  of  this  report  was  submitted  for  review 
and  comment  to  18  members  of  our  Procurement  Advisory 
Panel.  The  Panel  is  comprised  of  top  management  officials 
from  industry  and  the  academic  community.  Changes  were  made 
to  the  report,  where  appropriate,  to  reflect  their  comments. 
They  fully  support  the  need  for  more  emphasis  on  ownership 
considerations  in  the  design  of  weapon  systems  but  are 
concerned  that  the  funds  needed  to  bring  about  the  desired 
improvements  may  not  be  provided,  particularly  considering 
the  national  problem  of  funding  an  adequate  DOD  program. 

A  summary  of  other  key  comments  made  by  Panel  members  is 
included  in  appendix  IV. 
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HIGHLY  SOPHISTICATED  WEAPON  SYSTEMS  MUST  BE  DESIGNED 

WITH  USERS  IN  MIND 

America' s  leadership  in  technology  is  decades  old  and 
this  technology  has  provided  truly  advanced  defense  systems. 
Technology  has  also  introduced  new  challenges  to  weapon  system 
designers  to  assure  that  the  readiness  of  our  forces  is  main¬ 
tained  at  high  levels.  All  the  design  participants,  however, 
have  not  kept  pace.  A  tank  hatch  that  a  soldier,  clothed 
for  winter,  cannot  fit  through;  a  major  shipboard  fire  control 
system  that  cannot  be  adequately  supported;  aircraft  test 
equifxnent  that  causes  more  problems  than  it  solves;  and 
a  handheld  missile  that  when  fired  startles  the  person  that 
fires  it,  resulting  in  misses,  are  some  examples  of  the  prob¬ 
lems  with  currently  fielded  weapon  systems.  However,  it  is 
difficult  to  pinpoint  the  degree  to  which  system  effectiveness 
has  been  impaired  due  to  lack  of  attention  to  system  designs 
and  ownership  considerations.  What  is  needed  is  a  more  bal¬ 
anced  approach  in  designing  weapon  systems  which  gives  full 
and  adequate  attention  to  ownership  considerations. 

Although  increased  policy  emphasis  has  recently  been 
given  to  ownership  considerations,  the  effect  will  be  mean¬ 
ingless  unless  new  policies  are  properly  implemented.  We 
believe  there  are  three  important  ownership  factors  in  the 
acquisition  process  which  recent  history  suggests  are  among 
the  most  prominent  detractors  from  the  effectiveness  of 
deployed  systems — logistic  support,  human  reliability,  and 
quality  assurance.  Our  selection  of  these  ownership  factors 
for  analysis  does  not  imply  that  others  are  unimportant. 
Rather,  we  suggest  that  there  has  been  an  imbalance  of  funding 
and  attention  given  between  the  measurable  characteristics 
of  weapon  system  development  (cost,  schedule,  and  performance) 
and  these  other  factors  which  significantly  influence  the 
eventual  effectiveness  of  the  system  in  the  field. 

COMPLEXITY/SOPHISTICATION 

The  terms  complexity  and  sophistication  in  relation¬ 
ship  to  weapon  systems  are  in  themselves  somethipg  of  an 
enigma.  We  did  not  find  any  formal  definition  of  these 
terms  within  DOD  nor  did  we  find  a  finite  technique  for 
quantifying  system  complexity  or  sophistication.  What 
is  complex/ sophisticated  to  the  operator  or  the  maintenance 
crew  of  tanks  may  not  be  for  the  engineers  who  designed 
and  fabricated  the  tank,  or  to  the  program  manager  who 
managed  the  development  effort. 
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In  our  attempts  to  establish  the  relationship  between 
the  terms  "complexity/ sophistication"  and  the  operational 
availability  of  military  systems,  we  identified  many  types 
of  military  equipment  which  were  reported  to  be  undependable 
and/or  difficult  to  support  and  operate.  These  vary  from 
simple  infantry  entrenching  tools  (for  example,  shovels) 
which  did  not  fit  in  their  carrying  pouches,  through  boilers 
which  could  not  be  properly  maintained,  to  aircraft  avionics 
that  failed  because  of  software  program  deficiencies.  (For 
examples,  see  app.  V.) 

Some  people  measure  complexity  by  the  number  of  component 
parts  while  others  allude  to  the  relationship  and  inter¬ 
dependencies  of  various  components  and/or  subsystems.  Some 
understand  sophistication  to  mean  the  state-of-the-art  status 
of  a  component.  Others  use  sophistication  interchangeably 
with  complexity. 

We  believe  that  in  defining  the  terms  complexity/ 
sophistication  in  relation  to  weapon  systems  one  must 
include  such  factors  as  logistical  support,  human  factors, 
quality  assurance,  and  even  the  conditions  under  which 
hardware  must  be  employed  (for  example,  day  versus  night 
time) . 

TECHNOLOGY  IS  A  DRIVING  FORCE 


The  demand  for  high  performance  has  forced  designers 
to  incorporate  new  technology  into  systems  often  before 
its  reliability  has  been  fully  assessed.  On  the  design 
table  reliability  is  sometimes  compromised  in  favor  of 
performance  and  cost.  When  this  happens,  the  cost  in 
field  repairs  and  low  system  readiness  rates  can  be  high. 

In  the  commercial  sectot,  firms  tend  to  rely  on 
evolutionary  product  improvements.  Quantum  changes  occur 
only  when  technology  advances  have  been  proven.  Product 
improvements  are  generally  made  to  correct  specific  problems 
in  the  design  or  manufacturing  process,  and  the  impact  of 
these  changes  on  reliability  and  quality  are  evaluated. 

They  recognize  that  design  is  an  iterative  process  and 
seldom,  if  ever,  will  they  produce  a  perfect  design  the 
first  time,  even  though  they  incorporate  all  currently 
known  techniques . 

In  contrast,  DOD  tends  to  push  state-of-the-art 
advances  in  many  areas  simultaneously.  We  can  see  examples 
of  this  in  major  subassemblies  of  systems,  such  as  the 
automatic  test  equipment  on  the  F-15;  the  Tube- launched, 
Optically-Tracked,  Wire  Command  Link  (TOW)  Missile  System 
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on  the  COBRA  helicopter;  the  Mark  (MK)-86  Fire  Control  System 
on  Navy  ships;  and  the  turret  of  the  M60A2  tank.  Specifi¬ 
cally: 


— The  F-15  relies  on  intricate  electronics  to  survive 
and  effectively  accomplish  its  mission.  In  our 
review  we  examined  the  automatic  test  equipment  used 
to  support  the  F-15,  since  this  is  known  to  affect 
the  aircraft's  poor  feadiness.  The  software  used 
in  three  different  levels  of  equipment  was  incompat¬ 
ible  and  the  built-in  test  and  avionics  intermediate 
shop  equipment  was  unreliable.  Without  modifications, 
it  seems  doubtful  that  the  F-15  fleet  readiness  can 
appreciably  improve.  This  is  compounded  by  the  well- 
publicized  F-15  engine  reliability  and  durability 
problems.  (See  our  report,  "Are  Management  Problems 
In  the  Acquisition  of  Aircraft  Gas  Turbine  Engines 
Being  Corrected?,"  PSAD-80-72,  Sept.  30,  1980,  which 
attributes  the  engine  problems  to  inadequate  definition 
of  the  engine's  usage,  emphasis  on  performance  require¬ 
ments,  and  inadequate  flight  testing.) 

— The  Army's  only  deployed  attack  helicopter,  the  AH-1 
"COBRA,"  has  a  serious  maintainability/reliability 
problem  according  to  users  in  deployed  Army  units. 

One  of  the  helicopter's  primary  missions  is  killing 
tanks  using  its  TOW  missile  system.  The  subsystem 
used  in  launching  and  guiding  the  TOW  missiles  is 
experiencing  only  about  100  hours  mean  time  between 
failure  of  critical  mission-related  components.  The 
root  cause  of  these  failures  is  attributable  to  the 
poor  reliability  of  various  electronic  modules  and 
the  system's  built-in  test  equipment. 

— The  Navy's  MK-86  fire  control  system  is  the  primary 
weapon's  control  aboard  the  most  advanced  combat 
ships,  providing  multiple  modes  of  operation  and 
simultaneous  tracking  of  more  than  one  target.  It 
is  important  to  note  that  when  the  system  is  inoper¬ 
able,  the  ship  is  virtually  defenseless.  In  1979 
the  system  experienced  an  operational  availability 
rate  of  only  about  60  percent  despite  special  supply 
support  efforts.  The  primary  reasons  for  this 
low  availability  was  the  large  number  of  random  fail¬ 
ures  among  the  40,000  plus  parts  in  the  system  and 
the  extreme  difficulties  of  the  supply  system  to 
stock  sufficient  quantities  of  replacement  com¬ 
ponents  to  meet  the  demands  in  a  timely  manner. 

Also,  there  is  a  long  learning  curve  for  repair  tech¬ 
nicians  because  of  the  system's  complexity. 
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THE  AIR  FORCE'S  F-15  AIRCRAFT  HAS  EXPERIENCED  EXTENSIVE  PROBLEMS 
WITH  BOTH  ITS  AUTOMATIC  TEST  EQUIPMENT  AND  ITS  ENGINES 

SOURCE  DEPARTMENT  OF  THE  AIR  FORCE 


SOURCE  U  S  DEPARTMENT  OF  THE  ARMY 


CONTROL  SYSTEM,  WHICH  IS  OFTEN  NOT  OPERATIONAL 


— The  Army's  M60A2  tank  can  fire  either  a  152-mm.  pro¬ 
jectile  or  a  missile  through  the  same  tube.  The 
turret  on  this  tank  has  been  described  by  one  M60A2 
unit  commander  as  "fantastically  complex."  The  tank 
has  a  long  history  of  unreliability.  Operation  of 
the  tank  is  so  difficult  that  one  organization  in 
Europe  has  printed  a  detailed  checklist  of  sequential 
actions  to  be  accomplished  by  the  crew  before  driving 
and  firing  the  weapon.  We  examined  one  subsystem 
of  the  turret,  specifically  the  laser  rangefinder, 
and  found  electronic  reliability  problems. 

A  comment  by  General  Bernard  W.  Rogers,  former  Chief  of 
Staff,  U.S.  Army,  before  the  Senate  Armed  Services  Committee 
in  the  fiscal  year  1980  hearings  succinctly  describes  the 
situation  we  see. 

"Constant  striving  to  achieve  technological  excel¬ 
lence  is  causing  undue  technical  complexity,  lead¬ 
ing  to  high  cost  and  long  gestation.  Worse,  we 
have  come  to  assume  in  this  country  that  increases 
in  military  performance  come  only  from  technology 
and  that  the  purpose  of  technology  is  to  inprove 
upon  the  various  physical  characteristics  of 
familiar  weapon  systems.  The  result  is  a  failure 
to  anticipate  shifts  in  military  requirements, 
aggravated  by  long  gestation  periods  in  weapons 
development. " 

Dr.  William  J.  Perry,  Under  Secretary  of  Defense  for 
Research  and  Engineering,  also  recognizes  an  overreliance 
on  technology.  In  his  statement  before  the  Senate  Armed 
Services  Committee  on  March  14,  1979,  he  stated  that  a  dan¬ 
gerous  communications  gap  has  developed  between  the  developer 
of  systems  and  the  user.  This  gap  has  led  to  systems  that 
are  largely  technology  driven  and  are  poorly  united  to  the 
operational  need  because  the  user  did  not  know  how  to 
state  his  need  in  terms  of  the  available  technology.  He  also 
stated  that  DOD  research  and  development  programs  have  applied 
technology  to  enhance  performance  without  adequate  considera¬ 
tions  of  its  inpact  on  the  user  in  terms  of  support  costs 
and  the  nuniber  of  skill  levels  of  U.S.  military  personnel. 

The  results,  according  to  Dr.  Perry,  have  been  visible  in 
a  nunO^er  of  operating  systems  with  low  readiness  and  the 
need  for  expensive  retrofits  and  modifications. 


Private  industry  perceives 
problems  with  POP's  approach 

General  Rogers  and  Dr.  Perry's  concern  about  DOD's  over¬ 
reliance  on  technology  is  not  without  foundation.  In  our 
discussions  with  representatives  of  major  defense  contractors, 
we  surfaced  several  professional  opinions  that  reflect  upon 
the  life-cycle  reliability  of  military  systems.  These  opin¬ 
ions  indicate  that  DOD  does  practice  the  philosophy  expressed 
by  General  Rogers.  Two  of  the  major  points  raised  by  indus¬ 
try  follow. 

— DOD  tries  to  incorporate  too  many  "unproven"  technical 
approaches  in  new  systems .  We  were  told  that  a 
single  state-of-the-art  component  or  subassembly 
in  a  new  system  might  be  feasible  in  that  it  could 
be  watched  and  helped  to  mature.  However,  multiple 
state-of-the-art  innovations  are  not  practical 
because  they  cannot  be  adequately  monitored  and 
fixed  and  normally  tend  to  produce  an  unreliable 
system. 

— The  common  approach  used  by  DOD  and  the  defense 
industry  to  compensate  for  poor  human  aptitudes, 
inadequate  training,  and  reduced  numbers  of  personnel 
is  to  incorporate  more  automation.  In  effect,  the 
reliance  upon  man  is  being  phased  out  of  a  system 
by  using  built-in  test  equipment;  removable/replaceable 
modules;  and  more  reliance  on  returning  modules  for 
factory,  depot/general,  support-type  repair  (as  opposed 
to  diagnosis  and  repair  at  military  units) .  This  ap¬ 
proach  leads  to  rather  complex  electronic  components 
and  can  increase  the  number  of  "black  boxes”  needed 
in  the  supply  system  to  keep  a  system  operational. 

MAINTAINING  THE  BALANCE 

In  the  following  chapters  we  will  discuss  some  of  the 
changes  necessary  in  the  early  design  phase  of  an  acquisition 
(for  example,  logistics,  human  factors,  and  quality  that  must 
become  critical  design  considerations).  In  the  past  this 
has  not  generally  been  the  case.  In  fact,  we  were  told 
that  for  years  these  considerations  have  been  essentially 
"locked  out"  of  the  design  process.  (See  p.  12.)  The 
pressures  to  attain  specific  performance  goals  within  tight 
time  and  development  cost  constraints  have  led  management 
to  trade-off  or  otherwise  sacrifice  the  ownership  considera¬ 
tions  to  meet  short  term  budget  and  schedule  pressures. 

This  must  change  if  we  are  to  have  more  effective  systems. 
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Of  equal  importance,  however,  is  what  happens  throughout 
the  entire  acquisition  process  to  the  implementation  of 
these  considerations  by  management  under  the  pressures  of 
a  rapidly  changing  weapon  system  program.  The  program  manager 
is  constantly  being  confronted  by  the  need  to  make  trade-off 
decisions  that  affect  the  high  visibility  aspects  of  a  pro¬ 
gram.  In  this  environment,  ownership  considerations  can 
easily  be  traded-off  for  short  term  solutions  which  favor 
those  high  visibility  aspects.  No  matter  how  well  the  owner¬ 
ship  factors  have  been  designed  into  a  system,  they  will 
not  be  fully  effective  unless  management  is  motivated  to 
better  ensure  their  consideration  during  the  entire  process. 


FIGURE  I 


WEAPON  SYSTEM 
DESIGN  ROOM 


•  COST  SSCHEDULE  aPERFORMANCE 
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To  help  all  levels  of  management  give  greater  emphasis 
in  their  thinking  toward  the  ownership  considerations,  it 
might  be  necessary  to  change  the  way  in  which  the  procurement 
community  thinks  about  cost  (research  and  development,  pro¬ 
curement,  and  support),  schedule  (initial  operational  capa¬ 
bility  (IOC)),  and  performance  (speed,  weight,  firepower, 
and  so  forth).  For  example,  cost  should  be  the  life-cycle 
cost  of  an  acquisition,  schedule  should  be  the  time  when 
a  fully  supportable  system  is  fielded,  and  performance  should 
be  a  measure  of  a  system's  readiness  to  do  the  job  for  which 
it  was  designed. 

RECOMMENDATION  TO  THE  CONGRESS 

The  Congress  can  significantly  influence  the  direction 
DOD  takes  regarding  developing  weapon  systems  which  are 
operable,  maintainable,  and  supportable.  in  view  of  the 
problems  being  experienced  with  deployed  weapon  systems 
and  questions  concerning  the  readiness  of  our  Armed  Forces, 
it  is  particularly  important  that  such  emphasis  be  exerted 
now.  Accordingly,  we  recommend  that  the  Congress  direct 
more  attention  during  its  deliberations  on  DOD’ s  budget 
to  such  matters  as  logistic  support,  human  factors,  and  qual¬ 
ity  assurance  considerations  in  the  design  and  development 
of  weapon  systems. 
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CHAPTER  3 


LOGISTIC  SUPPORT — A  \ 

VITAL  DESIGN  CONSIDERATION  NOT  ADEQUATELY  ADDRESSED 

"*  *  *  Past  Defense  Reports  have  emphasized  unreli¬ 
able  and  hard-to-support  equipment  designs  as  a 
major,  and  often  the  principal,  contributor  to 
less-than-desirable  weapon  system  performance  in 
the  field." 

These  words  from  the  Secretary  of  Defense's  annual  report 
for  fiscal  year  1980  highlight  both  the  need  for  designing 
logistic  support  into  military  systems  and  the  failure  to 
satisfy  that  need.  DOD  has  been  trying  to  implement  some 
changes  in  the  acquisition  process  to  address  this  situation 
and  bring  the  process  more  into  balance  with  its  needs. 

There  are,  however,  many  problems  to  be  overcome  which  could 
severely  curtail  the  positive  impact  that  these  policy  changes 
need  to  have.  These  problems  include:  (1)  difficulties  in 
doing  logistic  support  analyses  (LSA),  (2)  difficulties  in 
doing  quantitative  analyses  for  projecting  readiness  rates 
of  various  system  designs. and  for  justifying  system  design 
investments  and  trade-offs  to  improve  logistic  support ability, 

(3)  lack  of  guidance  on  addressing  logistic  support  require¬ 
ments  in  critical  program  documents,  (4)  insufficient  evalua¬ 
tion  and  testing  of  the  integrated  logistic  support  (ILS) 
planning  and  logistic  supportability  of  systems,  and  (5) 
a  complacent  attitude  on  the  part  of  decisionmakers. 

LOGISTIC  SUPPORT  CONSIDERATIONS  MUST 
INFLUENCE  DESIGN 


Logistic  support  and  the  planning  for  it  need  to 
be  serious  considerations  in  the  design,  development,  and 
production  of  defense  systems.  All  systems  depend  upon 
logistic  support  to  create  and  sustain  their  effectiveness. 
The  design  capability  of  a  system  can  only  be  realized  if 
the  parts,  tools,  test  equipment,  personnel,  facilities, 
fuel,  and  so  forth  (that  is,  logistic  support)  are  available 
when  needed.  The  cost  of  providing  this  support  for  a  system 
often  exceeds  the  development  and  procurement  costs.  The 
high  cost  of  support  can  be  seen  in  the  fiscal  year  1981 
DOD  budget  requests  with  about  $60  billion — 37  percent 
of  the  total  budget — programed  for  logistics. 

To  acquire  a  weapon  system  which  can  be  effectively 
and  economically  supported,  there  must  be  continuing  inter¬ 
actions  between  the  logistician  and  the  design  engineer 
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throughout  the  acquisition  process.  This  is  particularly  im¬ 
portant  during  the  early  phases  of  design  where  the  stage  is 
set  for  the  bulk  of  a  system's  life-cycle  costs.  Many  studies 
of  life-cycle  and  weapon  system  supportability  show  that 
most  of  a  system's  life-cycle  costs  are  determined  during 
formulation  of  concepts  prior  to  milestone  I  of  the  weapon 
system  acquisition  process.  {See  fig.  2.) 

FIGURE  2 

SCHEDULE  OF  DECISIONS  AFFECTING  LIFE  CYCLE  COST 


SOURCE:  ProcMdin^i  from  National  Security  Industrial  Association  Syrnposium  on  Navy  Systems  Acquisition.  October  27  ?8.  1977 

Impact  of  not  considering 
logistic  support  in  design 

Some  acquisition  experts  consider  the  above  percentages 
to  be  quite  high  and  possibly  misleading.  They  point  out  that 
decisions  made  after  milestone  II  are  vitally  important  in 
determining  the  supportability  of  a  system  and  should  not  be 
underestimated.  Nevertheless,  we  found  near  universal  agree¬ 
ment  that  the  decisions  made  in  the  early  stages  of  the  ac¬ 
quisition  process  offer  the  greatest  opportunity  to  influence 
a  system's  supportability.  It  is  during  this  time  that 
recognition  of  the  consequences  of  the  design  on  availability 
and  supportability  can  lead  to  inexpensive  changes  to  the 
design  and  improve  eventual  operational  availability  and  sup¬ 
portability.  As  development  proceeds  and  the  design  becomes 
more  set,  changes  to  improve  supportability  become  more 
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difficult  and  costly  to  make.  Figure  3  from  Military 
Standard  1388,  "Logistic  Support  Analysis,"  illustrates 
our  point . 


Use  of  LSA  in  design 


To  promote  early  and  continuing  involvement  between  the 
design  engineer  and  the  logistician,  a  process  known  as 
LSA  has  been  established.  The  purpose  of  the  LSA  process  is 
to  inject  support  criteria  into  weapon  system  designs  and 
acquisitions  and  to  define  overall  system  support  require¬ 
ments.  LSAs  are  to  be  done  by  the  contractor  and  the 
effects  of  alternative  designs  are  to  be  considered.  Known 
scarcities,  constraints,  or  logistics  risks  are  to  be  iden¬ 
tified  and  methods  for  overcoming  or  minimizing  these 
problems  developed.  As  the  program  progresses  and  designs 
become  fixed,  the  LSA  process  concentrates  on  providing 
timely,  valid  data  for  all  areas  of  logistic  support  (for 
example,  maintenance,  provisioning,  personnel  and  training, 
and  technical  publications). 

IN  THE  PAST — LOGISTIC  SUPPORT 
SUFFERED  FROM  INATTENTION  IN  DESIGN 


Since  1964  DOD  policy  directives  have  formally  rec¬ 
ognized  the  need  to  give  attention  to  logistic  support 
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requirements  throughout  the  acquisition  process.  In  the 
intervening  years,  however,  the  approach  to  designing  sup¬ 
portable  systems  has  consisted  principally  of  specific  iso¬ 
lated  efforts  or  analyses,  not  the  systematic  comprehensive 
approach  of  the  LSA  process.  In  fact,  the  LSA  process  has 
rarely  been  initiated  in  the  design  stages  of  system  develop¬ 
ment  as  intended  and,  thus,  has  had  little  or  no  effect 
on  system  design. 

When  logistic  support  is  not  adequately  addressed  in  the 
design  process,  the  systems  often  suffer  from  low  states  of 
readiness  when  deployed  because  the  military  supply  and  main¬ 
tenance  systems  cannot  satisfactorily  meet  the  weapon  system's 
repair  needs.  We  found  numerous  examples  of  this  including 
the  Amy's  Forward  Area  Alerting  Radar  System  and  the  Navy's 
MK-86  Fire  Control  System.  In  each  case,  the  logistic  sup- 
portability  of  the  design  did  not  receive  adequate  review 
with  the  radar  not  being  subjected  to  a  logistic  demonstra¬ 
tion  test  and  the  fire  control  system  not  being  subjected 
to  a  maintenance  engineering  analysis.  These  are  critical 
steps  in  the  acquisition  process  to  ensure  the  logistic 
supportability  of  the  design.  Both  systems  have  since 
encountered  supportability  problems  that  can  be  traced  to 
the  inadequate  design  evaluation. 

Several  interrelated  factors  have  been  cited  as  reasons 
for  logistic  support  considerations  having  little  impact 
on  equipment  design.  Principally,  as  discussed  in  chapter 
2,  the  problem  has  been  that  the  acquisition  process  has 
focused  on  achieving  technical  performance  parameters 
(speed,  range,  firepower,  and  so  forth)  within  tight  acquisi¬ 
tion  cost  and  schedule  constraints  with  logistic  supportabil¬ 
ity  being  a  "consideration."  Two  important  contributors 
to  this  situation  have  been  the  difficulties  in  doing  LSAs 
in  the  design  stage  and  difficulties  in  quantifying  the 
benefits  of  logistic  support  design  features. 

Difficulties  getting  data  needed  to 
conduct  LSAs 


The  design  stage  of  the  acquisition  cycle  is  charac¬ 
terized  by  change  upon  change  as  the  engineers  attempt  to 
produce  designs  which  meet  many  requirements.  Because  of 
this  dynamic  state,  the  LSA  process  is  often  difficult  to 
initiate  in  the  very  early  stages  of  design.  Also,  one 
of  the  most  important  LSA  tasks  in  these  early  stages  is 
to  review  historical  data  to  relate  past  experiences  to 
the  design  requirements  of  new  acquisitions.  Information 
such  as  failure  rates,  major  support  cost  drivers,  and  repair 
time  on  like  or  similar  items  is  to  be  used  to  provide  a 
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basis  for  establishing  qualitative  and  quantitative  require¬ 
ments  on  new  equipment.  We  found  that  information  of  this 
nature  has  been  very  difficult  to  obtain.  Existing  DOD  main¬ 
tenance  data  reporting  systems  do  not  contain  the  information 
needed  and,  thus,  special  efforts  must  be  initiated  to  get 
it . 

Difficulties  quantifying  benefits 

It  is  very  difficult  to  quantify  the  benefits  of 
design  features  which  improve  the  logistic  supportability 
of  a  system.  Without  such  quantification,  there  is  little 
motivation  for  the  program  manager  and  contractor  to  raise 
development/production  costs  by  a  “known"  amount  to  save 
2U1  "unknown"  amount  of  money  in  future  support  costs. 

THE  PRESENT — DOD  IS  ATTEMPTING 
TO  EMPHASIZE  LOGISTIC  SUPPORT 


DOD  has  responded  to  this  need  by  more  clearly  defining 
and  emphasizing  the  role  of  logistic  support  in  acquiring 
defense  systems.  Current  initiatives  include 

— revising  DOD  acquisition  policies, 

— giving  more  visibility  to  logistics  matters  during 
Defense  Systems  Acquisition  Review  Council  (DSARC)  1/ 
meetings, 

— improving  the  supportability  of  systems  through  early 
test  and  evaluation,  and 

— giving  additional  attention  to  logistic  matters  by  the 
services. 

Considering  past  problems  associated  with  obtaining  the 
necessary  visibility  for  logistics  support  considerations, 
these  current  initiatives  represent  a  major  step  forward. 


1/DSARC  serves  as  an  advisory  body  to  the  Secretary  of  Defense 
on  the  acquisition  of  major  defense  system  progreuns  and 
related  policies  and  provides  him  with  supporting  informa¬ 
tion  and  recoiranendatlons  when  decisions  are  necessary. 
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Office  of  the  Secretary  of  Defense 
policy  efforts  to  improve  logistic 
planning 

The  DOD  Directive  5000.1,  "Major  System  Acquisitions," 
and  DOD  Instruction  5000.2,  "Major  System  Acquisition 
Procedures,"  were  revised  in  March  1980.  The  DOD  Directive 
4100.35  concerning  development  of  integrated  logistics 
support  programs  was  also  revised  and  reissued  in  January 
1980  as  DOD  Directive  5000.39,  "Acquisition  and  Manage¬ 
ment  of  Integrated  Logistic  Support  for  Systems  and 
Equipment."  The  revised  policies  stress  the  importance 
of  logistic  support  in  designing  new  systems  and  the 
need  for  logistic  support  planning  very  early  in  the 
acquisition  process. 

From  a  design  standpoint,  the  DOD  Directive  5000.1 
emphasizes  that  logistic  supportability  is  to  be  a  design 
requirement  as  important  as  cost,  schedule,  and  performance. 
Continuous  interface  between  the  program  office  and  the 
manpower  and  logistics  communities  is  to  be  maintained 
throughout  the  acquisition  cycle.  This  is  a  major  change 
from  past  policy,  which  usually  subjugated  logistic  support 
to  performance,  schedule,  and  cost. 

Whereas  DOD  Directive  5000.1  places  increased  im¬ 
portance  on  logistic  support  in  the  acquisition  process, 

DOD  Instruction  5000.2  generally  establishes  how  this  is 
to  be  accomplished.  Early  attention  to  logistic  support 
begins  at  milestone  0  with  the  establishment  of  logistic 
constraints  in  the  Mission  Element  Need  Statement  (MENS). 

The  intent  of  the  MENS  process  is  to  identify  needs  and 
then  explore  possible  alternative  solutions .  The  important 
consideration  from  the  support  viewpoint  is  that  MENS  pre¬ 
sents  an  opportunity  to  influence  the  alternative  selected 
from  the  very  beginning. 

In  addition  to  including  logistics  constraints  in 
MENS,  DOD  Instruction  5000.2  addresses  other  key  actions 
designed  to  further  emphasize  supportability  and  influence 
system  design.  Beginning  early  in  the  system  development 
process,  both  DOD  and  industry  should  consider  innovative 
manpower  and  support  concepts.  When  competitively  obtaining 
alternative  concept  solutions  to  mission  needs,  the  widest 
possible  range  of  acquisition  and  support  alternatives  to 
satisfy  the  mission  need  is  to  be  considered.  Furthermore, 
readiness  problems  and  support  cost  drivers  of  current  systems 
are  to  be  analyzed  to  identify  potential  areas  of  improvement 
i  to  be  addressed  during  concept  formulation. 

[ 
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Directive  5000.39  reemphasizes  the  need  for  an  ILS 
program  that  begins  at  milestone  0.  To  do  this,  the  direc¬ 
tive  requires  that  program  budgets  include  adequate  funding 
for  ILS  planning,  analysis,  and  cost  reduction  efforts. 

The  ILS  planning  must  be  based  on  the  constraints  identified 
at  milestone  0,  the  deployment  concept,  system  readiness 
objectives,  realistic  maintenance,  related  reliability 
and  maintainability  estimates,  and  the  documented  LSA.  Of 
particular  note  here  is  the  importance  given  to  LSA  in  ILS 
planning.  For  the  first  time,  LSA  is  a  requirement  and 
is  to  begin  at  milestone  0. 

Other  major  points  in  DOD  5000.39  include  the  use  of 
ILS  goals  and  objectives  in  acquisition  strategic  planning 
and  contract  incentives;  the  establishment  of  additional 
ILS  research  and  study  programs;  and  a  clear  "audit  trail" 
of  changes  in  support  budgets,  support  related  goals,  and 
thresholds  including  changes  in  definition.  Also,  industry 
innovation  in  support  alternatives  is  encouraged. 

Attention  to  logistics 
matters  by  PS ARC 

The  revised  policies  require  the  preparation  of  more 
detailed  logistic  support  information  for  DSARC  reviews 
and  describe  the  logistic  support  activities  which  should 
be  accomplished  before  each  major  milestone  decision. 

The  composition  of  DSARC  also  provides  some  assurance 
that  logistics  factors  will  be  addressed.  The  Assistant 
Secretary  of  Defense  (Manpower,  Reserve  Affairs,  and  Logis¬ 
tics)  is  a  permanent  member  of  DSARC  and  is  responsible 
for  ensuring  that  logistic  planning  is  consistent  with  system 
hardware  parameters,  logistic  policies,  and  readiness  objec¬ 
tives.  Assisting  in  this  effort  is  the  ILS  Analysis  Division 
which  reviews  major  acquisition  programs  from  a  logistic 
management  standpoint  as  they  become  available  for  DSARC 
review.  One  of  the  primary  assessments  made  is  a  quantita¬ 
tive  analysis  to  project  system  readiness  levels. 

Testing  for  logistic  support 

In  March  1979  DOD  initiated  a  two-phased  study  of  the 
weapon  system  test  and  evaluation  process  related  to  opera¬ 
tional  suitability,  including  logistic  supportability . 

The  first  phase  of  the  study  effort  found  that  the 
requirement  to  evaluate  operational  suitability  is  well 
established  in  top  level  policy  directives.  However,  guide¬ 
lines  for  implementation  do  not  exist  in  any  consolidated 
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form,  nor  does  existing  DOD  documentation  clearly  state 
to  what  extent  operational  suitability  will  be  addressed 
at  each  phase  of  the  program. 

The  study  further  observed  that  the  assessment  of 
system  operational  suitability,  under  any  conditions,  is 
a  complex  undertaking.  A  number  of  factors  and  conditions 
were  also  identified  which  make  the  process  more  difficult 
and  demanding.  Some  of  the  problems  center  around  test 
hardware  immaturity,  lack  of  representative  support  and 
test  equipment,  nonrepresentative  training,  artificial 
test  environments,  and  the  complex  nature  of  weapon  systems 
and  their  supporting  logistics  systems. 

In  addition  to  these  inherent  technical  problems,  many 
development  programs  have  been  compressed  to  meet  tight 
and  concurrent  schedule  objectives.  As  a  result,  the  time 
allotted  for  testing  to  evaluate  the  supportability  charac¬ 
teristics  of  systems  in  development  is  often  inadequate. 

The  second  phase  of  the  study  is  addressing  the  need  for 
revisions  to  current  testing  policies  and  procedures. 

Attention  to  logistics  matters 
by  the  services 

The  services  have  undertaken  several  initiatives  to 
ensure  the  consideration  of  logistic  support  factors  in 
the  acquisition  cycle.  Two  of  these  initiatives  are 
(1)  establishment  of  groups  to  review  the  status  of 
logistic  support  planning  for  ongoing  programs  and  (2) 
emphasis  on  implementing  LSAs. 

Logistic  review  groups 

Each  service  has  an  extensive  system  acquisition  review 
process.  This  process  extends  from  the  project  office  level 
up  to  and  including  the  office  of  individual  service  secre¬ 
taries.  Although  logistic  support  planning  is  addressed 
during  these  various  levels  of  review,  the  services  recently 
established  individual  logistic  review  groups  in  an  attempt 
to  give  some  additional  attention  to  logistics  considerations 
during  the  acquisition  process.  The  ability  of  these  groups 
to  affect  an  individual  program’s  logistics  effort  varies 
from  service  to  service. 

Additional  attention  being 

given  to  LSA 

Groups  specializing  in  applying  LSA  have  been  established 
in  each  of  the  services.  These  groups  are  available  to 
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assist  project  offices,  upon  request,  with  developing 
and  implementing  the  LSA  process  in  their  respective 
programs . 


Another  indication  of  the  emphasis  being  placed  on 
LSA  is  the  establishment  of  an  unofficial  joint  service 
LSA  working  group  in  late  1978.  At  this  time  the  logistic 
establishments  of  all  three  services  and  the  Marine  Corps 
met  to  discuss  the  status  of  LSA  implementation.  It  was 
decided  a  joint  service  effort  would  be  initiated  to  develop 
a  common  basis  for  LSA  implementation. 

Since  that  time  industry  representatives  have  joined 
the  group  and  a  joint  service  memorandum  of  understanding 
has  been  signed.  The  memorandum  of  understanding  defines 
how  the  services  will  work  together  to  increase  LSA's  effec¬ 
tiveness.  As  a  result,  the  Office  of  the  Secretary  of  Defense 
(OSD)  has  recently  agreed  to  formalize  the  working  group  and 
take  the  lead  in  directing  its  efforts.  One  of  the  first 
projects  to  be  undertaken  is  the  updating  of  Military  standard 
1388.  This  is  being  done  to  (1)  meet  today’s  system  engineer¬ 
ing  requirements,  (2)  more  clearly  explain  the  tailoring 
of  the  LSA  process,  and  (3)  assist  in  making  LSA  task  state¬ 
ments  more  explicit. 

THE  FUTURE— WILL  THE  SYSTEMS 
DEPLOYED  BE  SUPPORTABLE? 

While  most  everyone  we  interviewed  was  pleased  with 
the  new  policies,  many  were  also  skeptical  about  whether 
the  policies  will  be  implemented.  We  share  their  concern. 

The  problems  which  will  make  it  difficult  to  satisfactorily 
implement  the  new  policies  include 

— difficulties  in  doing  LSAs, 

— difficulties  in  projecting  readiness  rates, 

— inadequate  guidance,  particularly  on  LSA, 

— insufficient  evaluation  and  testing,  and 

— potentially  COTiplacent  attitudes  on  the  part  of 
decisionmakers . 

Difficulties  in  doing  LSAs 

The  three  key  difficulties  to  having  an  effective 
LSA  progreun  starting  at  milestone  0  are  (1)  getting 
the  information  needed  to  start  analyses,  (2)  getting 
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qualified  people  who  can  do  LSA,  and  (3)  getting  the  funds 
to  pay  for  LSA.  To  have  available  the  type  of  informa¬ 
tion  and  data  needed  on  existing  systems,  equipment,  and 
parts,  the  current  defense  data  reporting  procedures  will 
have  to  be  modified  or  new  ones  developed.  (See  pp.  17  and 
18.)  Until  such  changes  are  made,  special  data  gathering 
efforts  will  have  to  be  initiated  on  each  new  development 
program. 

The  shortage  of  qualified  people  to  do  LSA  is  also 
widely  recognized  within  DOD  and  by  contractors.  DOD 
training  programs  have  been  expanded  recently  to  try  to 
cope  with  the  problem,  but  it  appears  that  the  shortage  will 
exist  for  many  more  years.  Also,  as  noted  earlier,  the  serv¬ 
ices  have  established  offices  to  assist  project  offices 
and  contractors  with  their  LSA  progrzuns.  These  offices, 
however,  said  that  they  are  in  need  of  more  trained  staff 
members  especially  with  the  added  emphasis  now  being  placed 
on  LSAs . 

Because  of  the  difficulty  of  getting  data,  the  need 
for  qualified  people,  and  the  complexity  of  an  LSA  program 
which  can  require  elaborate  ccxnputer  software,  LSA  progreuns 
can  be  very  expensive.  Unless  DOD  is  willing  to  invest 
more  funds  in  the  early  research  and  development  efforts 
of  programs  to  pay  for  indepth  LSAs,  the  comprehensive 
LSA  efforts  needed  to  effect  design  changes  will  not  materi¬ 
alize.  The  amount  of  additional  funds  needed  can  be  partially 
offset  through  tailoring  LSA  to  the  specific  program  needs 
and  then  purging  separate  analyses  which  provide  only  dupli¬ 
cative  information  to  that  obtained  through  LSA.  We  found 
this  to  be  a  very  ccxnmon  problem.  Also,  LSAs  can  often 
be  made  simpler  and  less  expensive  by  using  relatively 
simple  models  complemented  with  a  good  knowledge  of  design 
alternatives . 

Difficulties  in  projecting 
readiness  rates 


The  quantitative  analyses  called  for  in  Directive  5000.39 
to  assess  the  adequacy  of  logistic  plans,  resources,  and 
support- related  par6uneters  to  meet  system  goals  will  also 
be  very  difficult  to  perform.  The  major  problems  will  be 
obtaining  the  necessary  data  inputs  and  developing  valid 
models  to  use  in  projecting  the  probable  system  readiness 
rates.  There  are  many  models  available,  but  their  usefulness 
in  making  accurate  readiness  predictions  is  questionable. 
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Inadequate  guidance 

There  has  been  and  continues  to  be  a  shortage  of 
guidance  to  ensure  that  critical  initiating  program 
documents — particularly  MENS,  requests  for  proposals, 
and  contracts — contain  the  language  needed  to  get  logistic 
support  designed  into  systems .  The  MENS  statements  are 
to  contain  logistics  considerations,  but  only  the  Air 
Force  has  issued  detailed  guidance  to  assist  those  who 
prepare  MENS  to  ensure  that  logistic  considerations  are 
effectively  addressed.  Guidance  is  also  needed  on  how 
to  incorporate  qualitative  and  quantitative  requirements 
in  contracts  which  force  designers  to  give  attention  to 
supportability . 

Also,  there  is  very  little  guidance  provided  by  the 
services  to  project  offices  on  how  to  implement  LSA  in 
contracts.  Guidance  is  needed  on  how  to  prepare  requests 
for  proposals  and  contracts  so  that  the  contractor's  LSA 
program  will  be  tailored  to  the  needs  of  the  system  to 
be  acquired.  Criteria  is  also  needed  for  evaluating  con¬ 
tractor's  LSA  program  proposals  for  source  selection  pur¬ 
poses.  A  standard  format  is  needed  for  contractors  to 
use  in  preparing  their  LSA  proposal  to  ensure  that  competing 
contractor's  proposals  can  be  compared  and  that  each  addres¬ 
ses  all  aspects  of  the  proposed  LSA  effort.  This  guidance 
is  important  because,  if  a  program  is  to  have  a  comprehensive 
LSA,  it  must  be  written  into  the  contract  as  it  is  the 
contractor  who  will  do  most  of  the  LSA.  While  Directive 
5000.39  contains  implementing  guidance,  we  believe  that 
this  needs  to  be  supplemented  with  detailed  guidance  (possi¬ 
bly  even  model  statements)  to  ensure  that  specific  matters, 
such  as  those  cited  above,  are  adequately  addressed. 

Insufficient  evaluation  and  testing 

Adequate  independent  evaluation  of  a  program’s  ILS 
planning,  testing,  and  evaluation  of  a  system’s  logistic 
supportability  before  a  production  decision  are  critical 
parts  of  the  acquisition  process.  Currently,  these  tasks 
are  not  being  adequately  accomplished  and  improvements 
are  needed. 

We  found  that  the  logistic  review  groups  in  the  Army, 
Navy,  and  Air  Force  (see  p.  21)  review  primarily  major 
progreuns  and  thus  many  less  than  major  programs  are  not 
subjected  to  the  type  of  Independent  review  needed  to 
ensure  that  logistic  support  is  adequately  planned.  The 
Naval  Material  Command  has  instructed  its  three  system 


24 


conunands  (Air,  Sea,  and  Electronics)  to  set  up  such  reviews, 
but  unless  additional  staff  members  are  added,  we  believe 
it  will  be  very  difficult  to  get  these  reviews  done. 

The  need  for  improved  testing  and  evaluation  of  a  system 
during  development  has  been  identified  by  OSD.  (See  pp.  20 
and  21.)  This  will  continue  to  be  a  serious  problem  until 
DOD  takes  steps  to  improve  the  quality  of  the  testing  program. 

Complacent  attitude  of  decisionmakers 

The  attitude  toward  logistic  support  which  has  fostered 
so  many  of  the  problems  the  Armed  Forces  have  today  may 
plague  the  implementation  of  the  new  policies.  Decisionmakers 
throughout  the  acquisition  process  will  have  to  be  willing 
to  invest  the  resources  needed  to  implement  the  new  policies. 

Management  will  still  be  driven  to  meet  cost,  schedule, 
and  technical  performance  pareimeters.  Increased  emphasis 
on  initiating  the  LSA  process  at  the  start  of  the  acquisition 
process  will  help,  but  LSAs  do  not  make  decisions.  If 
management  must  sacrifice  acquisition  dollars  or  technical 
performance  to  improve  supportability ,  there  could  be  little 
impetus  for  them  to  do  so,  especially  considering  the  extreme 
difficulties  in  quantifying  the  benefits  of  improved  support- 
ability.  As  a  consequence,  most  design  trade-offs  may  con¬ 
tinue  to  favor  cost,  schedule,  and  technical  performance. 

CONCLUSIONS 


In  the  past  there  has  been  insufficient  attention  given 
to  logistic  support  planning  during  the  acquisition  cycle. 

We  believe  this  has  contributed  to  the  poor  operational 
availability/supportability  of  many  deployed  systems.  These 
problems  along  with  upward  spiraling  operational  and  support 
costs  have  led  DOD  to  make  long  needed  revisions  to  major 
acquisition  policy,  including  making  supportability  as  im¬ 
portant  an  acquisition  consideration  as  cost,  schedule, 
and  performance.  Increased  attention  is  now  being  directed 
to  logistic  factors  at  key  acquisition  decision  points. 

Although  these  initiatives  represent  a  major  step  in 
the  right  direction,  successful  implementation  will  be  dif¬ 
ficult  because  of  (1)  the  difficulties  in  doing  LSAs,  (2) 
the  difficulties  in  doing  quantitative  analyses  for  project- 
i  ing  readiness  rates  of  various  system  designs  and  for  justify- 

[  ing  system  design  investments  and  trade-offs  to  improve  logis¬ 

tic  supportability,  (3)  the  lack  of  guidance  on  addressing 
logistic  support  requirements  in  critical  progr6un  documents. 
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(4)  insufficient  evaluation  and  testing  of  the  ILS  planning 
and  logistic  supportability  of  systems,  and  (5)  a  poten¬ 
tially  complacent  attitude  by  decisionmakers. 


Unless  these  needs  and  problems  are  addressed  by  DOD, 
the  positive  impact  of  the  new  policy  initiatives  will  never 
fully  materialize.  The  actions  needed  will  require  people 
and  money,  both  scarce  resources.  However,  we  see  this 
as  a  necessary  step  if  DOD  really  Intends  to  minimize  the 
problems  of  high  support  costs  and  low  readiness  rates  so 
common  in  today's  complex/ sophisticated  weapon  systems. 

RECOMMENDATIONS  TO  THE 
SECRETARY  OP  DEFENSE 


We  recommend  that  the  Secretary  of  Defense 

— develop  new  or  modify  current  data  reporting  proced¬ 
ures  to  provide  the  information  needed  for  performing 
LSAs , 

— establish  logistic  support  research  and  study  progreuns 
to  develop  improved  quantitative  methods  for  assessing 
(1)  the  adequacy  of  logistic  support  plans,  resources, 
and  support- related  parameters  and  (2)  the  benefits 
of  design  changes  in  the  acquisition  process  to  improve 
the  supportability  of  a  system, 

— provide  detailed  guidance  to  ensure  that  critical  pro- 
greun  documents  (for  exeunple,  MENS,  request  for  propos¬ 
als,  and  contracts)  contain  the  language  needed  to 
obtain  systems  which  are  designed  to  be  supportable, 
and 

— provide  for  improved  testing  and  evaluation  of  the 
supportability  of  systems  before  they  are  deployed. 

AGENCY  COMMENTS 


DOD  agreed  with  our  findings,  conclusions,  and  recom¬ 
mendations.  (See  app.  III.)  In  commenting  on  the  need  for 
logistic  support  research  and  study  programs,  OSD  stated 
that  direction  was  sent  in  September  1980  to  the  military 
departments  to  initiate  a  research  and  development  program 
for  Improvonent  of  weapon  system  support.  Regarding  the  need 
for  guidance,  DOD  pointed  out  that  some  implementing  guidance 
has  been  Included  in  DOD  Directives  5000.39  and  5000.40. 

This  guidance,  however,  is  not  in  the  detail  needed,  as  dis¬ 
cussed  on  page  24. 


CHAPTER  4 


HUMAN  RELIABILITY — A  MAJOR  FACTOR  WHICH  HAS  NOT 
BEEN  ADEQUATELY  CONSIDERED  IN  SYSTEM  DESIGN 

Failures  of  deployed  systems  are  often  caused  by  human- 
induced  errors.  There  are  indications  that  the  percentage 
of  failures  due  to  human  ineptitude  or  poor  human  reliabil¬ 
ity  may  be  quite  high.  The  increasingly  complicated 
nature  of  modern  military  systems  together  with  shortages 
of  qualified  military  personnel  suggests  that  human-induced 
errors  both  in  operation  and  maintenance  of  systems  will 
increase  unless  more  attention  is  given  to  this  problem 
in  the  design  and  development  phases  of  the  acquisition 
process . 

Military  specifications,  standards,  and  handbooks  on 
human  factors  do  not  adequately  consider  human  limitations 
such  as  skill  levels,  proficiency,  availability,  environmental 
stress,  and  fatigue.  Also,  there  are  no  common  methodologies 
or  data  sources  for  forecasting  skill  levels  of  future  mili¬ 
tary  personnel,  and  there  does  not  appear  to  be  sufficient 
emphasis  on  testing  systems  from  a  human  reliability  stand¬ 
point. 

HUMAN- INDUCED  MALFUNCTIONS  MAY 
BE  A  VERY  SERIOUS  PROBLEM 


The  human  limitations  of  the  operator  and  maintainer  can 
cause  system  failures.  The  extent  of  this  problem,  however, 
cannot  be  accurately  estimated  from  data  in  current  failure 
reporting  systems .  Based  on  interviews  we  conducted  and 
studies  we  reviewed,  we  are  convinced  that  a  large  number  of 
system  failures  are  the  result  of  human  error.  We  found  a 
DOD  and  contractor  study  which  share  this  position.  Both 
estimate  that  human  errors  account  for  at  least  50  percent 
of  the  failures  of  major  systems. 

The  problem  of  human-induced  failures  may  very  well  be¬ 
come  worse.  Attendant  to  the  increasingly  complicated  nature 
of  systems  are  the  lower  education  and  aptitude  levels  of 
personnel  now  entering  the  services,  the  shortages  and  high 
turnover  rate  of  experienced  personnel  which  leads  to  very 


^/Human  reliability  as  used  here  is  defined  as  the  probability 
that  human  error  (by  either  the  operator  or  the  maintainer) 
will  not  cause  a  system  failure  or  malfunction. 


A  LARGE  NUMBER  OF  WEAPON  SYSTEM  FAILURES  MAY  BE  THE  RESULT  OF 
HUMAN  ERROR. 
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low  overall  experience  levels,  and  the  effect  of  greater 
use  of  complex/sophisticated  automatic  checkout  and  built-in 
test  equipment.  For  a  system  to  operate  successfully, 
the  designer  must  fully  consider  the  following  types  of 
characteristics  of  the  personnel  who  will  operate  and 
maintain  the  system  when  it  is  eventually  deployed: 

— Muscular  strength  and  coordination. 

— Body  dimensions. 

— Perceptions  and  judgment. 

— Sensory  capacities. 

— Native  skills  and  capacity  to  learn  new  skills. 

— Optimum  workload. 

— Basic  requirements  for  comfort,  safety,  and  freedom 
from  environmental  stress. 

THE  IMPACT  OF  PEOPLE  ON  SYSTEMS 


The  following  quote,  taken  from  an  Army  technical  memo¬ 
randum,  indicates  the  importance  of  the  operator’s  and 
maintainer's  relationship  to  an  item  of  hardware. 

"People  are  the  only  responsible  agents  in  the 
system.  No  matter  how  small  the  roles  assigned 
to  people,  they  are  responsible  roles.  People 
determine  whether  the  system  is  ready  to  operate, 
what  it  is  to  do,  how  and  when  it  is  to  do  it, 
when  and  what  variations  in  performance  are  to 
occur,  and  what  constitutes  adequate  or  complete 
performance.  People  decide,  control,  guide, 
change,  and  evaluate.  They  are  expected  to 
anticipate,  detect,  compensate  for,  and  explain 
any  undesirable  variations  in  performance.  And 
their  errors  assume  a  significance  commensurate 
with  their  responsibilities." 

Although  human-induced  failures  adversely  affect  opera¬ 
tional  availability,  the  failures  are  usually  charged  to 
system  unreliability.  A  failure  or  an  unscheduled  delay 
is  considered  to  be  human  initiated  only  if  the  human  can 
be  clearly  identified  as  the  "causative  agent"  leading  to 
the  failure  or  system  inoperativeness.  Many  human-initiated 
malfunctions,  however,  are  not  clearly  identifiable.  This 
lack  of  recognition  is  not  an  easily  cured  malady.  The 
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hardware  failure  usually  occurs  before  the  human- induced 
action  is  detected  and  it  (the  failure)  is  a  quantifiable 
entity  making  it  easy  to  report-  In  general,  the  five  types 
of  human  errors  which  cause  most  failures  can  be  identified 
as  follows; 

— Failure  to  follow  procedures, 

— Incorrect  diagnosis  of  particular  situations. 

— Misinterpretation  of  communications  (written  or 
verbal ) . 

— Inadequate  support,  tools,  equipment,  and  environment. 

— Insufficient  attention  or  caution. 

HUMAN  RELIABILITY  CAN  BE  IMPROVED 
OR  DEGRADED  BY  DESIGN  OF  EQUIPMENT 

Poor  design,  particularly  of  support,  maintenance,  and 
checkout  equipment,  can  significantly  increase  the  probabil¬ 
ity  of  system  failures  once  the  system  is  deployed. 

The  following  very  brief  itemization  is  representative 
of  the  types  of  problems  that  can  be  directly  related  to  the 
design  of  the  system  or  its  support  equipment  compared  to 
the  capabilities  or  aptitudes  of  operators/maintenance 
personnel : 

— Indicator  meters  and  readouts  not  readily  visible. 

— Parts  not  accessible  or  special  tools  required. 

— Multiple-interactive  adjustments  required. 

— Visual  aids  and  wiring  diagrams  which  are  overly 
complex . 

— Labeling  and  coding  instructions  unclear. 

— Faulty  equipment  setup  awkward  for  operator  use  or 
maintenance  action. 


^/it  should  be  noted  that  human  nonadherence  to  procedures 
may  be  detrimental  to  system  performance  and  produce  mis¬ 
sion  failures,  but  these  procedural  errors  do  not  neces¬ 
sarily  induce  equipment  failures. 
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We  found  several  specific  examples  of  system  designs 
significantly  increasing  the  probability  of  human-induced 
errors.  On  the  P-3  and  S-3  aircraft  programs,  a  large 
nundaer  of  deficiencies  identified  when  these  systems  were 
first  deployed  were  identified  as  operator  errors  induced 
by  poor  design  of  the  aircraft’ s  display  and  control  com¬ 
ponents.  An  antitank  weapon  system  is  another  example. 

In  firing  the  weapon,  the  normal  operator  is  unable  to  keep 
still  long  enough  to  accurately  direct  the  missile  to  the 
target. 

We  believe  there  has  been  considerable  effort  to  adapt 
man  to  the  constraints  built  into  the  hardware  instead  of 
using  manpower  factors  as  design  criteria.  Until  OSD  pub¬ 
lished  the  memorandum,  "Manpower  Analysis  Requirements  for 
System  Acquisition,"  August  17,  1978,  there  was  no  specific 
DOD-wide  guidance  on  manpower  planning  for  new  systems . 

A  review  of  past  and  existing  military  specifications, 
standards,  and  handbooks  on  human  factors  and  human  engineer¬ 
ing  reveals  that  most  of  them  deal  exclusively  with  the 
human  physical  characteristics  and  design  interface.  Although 
they  furnish  a  basis  for  design  of  the  immediate  interface 
between  man  and  machine,  they  do  not  provide  the  broader 
manpower  factors  data  (for  exeunple,  skill  levels,  proficiency, 
availability,  rotation  rates,  cost,  and  so  forth)  necessary 
to  evaluate  alternative  designs  to  determine  the  optimum 
design  for  minimum  cost  of  ownership  and  maximum  effective¬ 
ness.  The  recently  revised  DOD  Instruction  5000.2,  March  19, 
1980,  does  address  skill  requirement  as  a  "consideration 
and  constraint"  in  system  design. 

INSUFFICIENT  EARLY  PLANNING 
FOR  HUMAN  RELIABILITY 


In  pursuing  the  question  of  why  deployed  systems  seem 
to  be  afflicted  by  so  many  human-related  problems,  we  found 
that  there  are  no  commonly  accepted  methodologies  or  sources 
of  data  which  the  services  can  use  to  forecast  skill  levels 
of  potential  military  personnel  in  the  upcoming  5-  to  10-year 
period.  Without  such  basic  data,  it  is  difficult  for  hardware 
developers  to  properly  estimate  human  reliability  and  consider 
it  in  the  design  of  a  system. 

The  need  for  improved  personnel  planning  data  has  been 
recognized  within  DOD.  In  1977  the  Navy  Initiated  the 
Hardman  program  to  develop  methodologies  for  determining 
manpower  requirements  associated  with  systems  being  developed 
and  procured.  In  February  1978  the  Army's  Operational  Test 
and  Evaluation  Agency  informed  the  Army  Vice-Chief  of  Staff 
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that  the  neglect  of  human  characteristics  in  planning  and 
testing  has  caused  serious  and  costly  problems .  In  August 
1978  the  Assistant  Secretary  of  Defense  (Manpower,  Reserve 
Affairs,  and  Logistics)  informed  the  secretaries  of  the 
military  departments  that  they  "should  place  greater  emphasis 
on  controlling  and  forecasting  the  effects  of  our  weapon 
systems  on  manpower  and  personnel  needs." 

In  November  1978,  the  Army  Materiel  and  Readiness 
Command  instructed  its  program  managers  and  development 
commands  to  prepare  a  human  factors  engineering  analysis 
for  presentation  at  the  preliminary  review  of  each  Army 
Selected  Acquisition  Review  Council  milestone.  Army's 
Human  Engineering  Laboratory  was  tasked  to  provide  assis¬ 
tance.  To  our  knowledge,  that  is  the  only  human  factors- 
dedicated  organization  which  now  has  a  more  or  less  direct 
access  (through  the  Office  of  the  Army's  Deputy  Chief  of 
Staff  for  Personnel)  to  the  acquisition  management  process. 

TESTING  PROCEDURES  HAVE  NOT  GIVEN 
HUMAN  RELIABILITY  ADEQUATE  EMPHASIS 


In  our  review  of  how  much  human  reliability  emphatio  is 
applied  during  testing,  we  received  the  following  from  the 
military  services  test  organizations. 

— In  many  cases,  dedicated  human  reliability  testing  is 
permitted  only  on  a  "not-to-interfere"  basis. 

— In  the  early  stages  of  system  testing,  participants 
are  usually  drawn  from  contractor  technicians,  engi¬ 
neers,  and  other  contractor  personnel.  This  practice 
could  easily  result  in  biased  outcomes  as  contractor 
personnel  are  often  more  skilled  and  experienced 
than  the  military  personnel  who  will  ultimately  use 
the  system. 

— When  human  reliability  has  been  acknowledged  during 
testing,  the  attention  was  normally  dedicated  to 
the  performance  of  equipment  operators  versus  main¬ 
tenance  personnel.  Maintenance  considerations  have 
generally  been  minimal,  if  considered  at  all.  In 
the  area  of  maintenance  the  concern  seems  to  be  in 
structuring  training  courses,  not  examining  designs 
for  things  like  accessibility,  degree  of  difficulty, 
and  so  forth. 

— Even  when  a  system  reaches  the  testing  stage,  program 
managers  do  not  get  easily  motivated  by  nonhardware 
limitations  presented  by  human  engineering. 
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Anthropometric  (man's  physical  size)  measurements 
are  always  considered  during  testing,  but  intangibles 
such  as  perception  limitations,  man's  performance 
under  stress  or  fatigue,  and  workload  limitations 
are  not  commonly  appreciated. 

CONCLUSIONS 


There  are  indications  that  human  ineptitude  or  poor  human 
reliability  may  cause  over  50  percent  of  all  weapon  system 
failures.  The  increasingly  complicated  nature  of  modern  mili¬ 
tary  systems  together  with  internal  military  personnel  prob¬ 
lems  suggests  that  human-induced  errors  both  in  operations 
and  maintenance  could  also  increase  unless  more  attention 
is  paid  to  this  problem  during  design  and  development.  Weapon 
system  designs  have  been  dictating  manpower  requirements. 

What  is  needed  is  a  continuing  interface  between  the  system 
designers  and  the  manpower  planners  with  manpower  requirements 
influencing  system  design  and  vice  versa. 

If  the  design  of  systems  is  to  adequately  consider  all 
the  human  limitations  (including  skill  levels,  proficiency, 
availability,  environmental  stress,  and  fatigue),  military 
specifications,  standards,  and  handbooks  must  address  these 
factors.  Existing  documents  do  not.  Also,  common  methodolo¬ 
gies  and  sources  of  data  are  needed  to  forecast  skill  levels 
of  potential  military  personnel  5  to  10  years  in  the  future. 
This  information,  which  would  be  extremely  valuable  to  system 
designers  and  testers,  is  currently  not  available. 

Finally,  there  does  not  appear  to  be  sufficient  emphasis 
on  testing  systems  from  a  human  reliability  standpoint  par¬ 
ticularly  in  the  developmental  stages  of  the  acquisition 
process.  This  could  result  in  design  errors  requiring  expen¬ 
sive  modifications  after  the  system  is  deployed. 

RECOMMENDATIONS  TO  THE 
SECRETARY  OF  DEFENSE 


We  recommend  that  the  Secretary  of  Defense 

— modify  human  t^^'^or  specifications,  standards,  and 

handbooks  used  in  system  design  so  that  they  adequately 
address  human  limitations,  such  as  skill  levels,  pro¬ 
ficiency,  availability,  environmental  stress,  and 
fatigue,  which  can  result  in  human-induced  system 
failures. 
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— develop  common  methodologies  and  data  sources  for 
use  by  system  designers  in  forecasting  skill  levels 
of  military  personnel  5  to  10  years  in  the  future,  and 

— ensure  that  all  major  systems  are  subjected  to  adequate 
testing  and  examination  from  a  human  factors  stand¬ 
point,  throughout  the  acquisition  process,  particularly 
in  the  developmental  stages . 

AGENCY  COMMENTS 


DOD  agrees  with  our  findings,  conclusions,  and  recom¬ 
mendations.  (See  app.  ill.)  In  commenting,  DOD  emphasized 
the  need  for  continuing  interaction  between  system  designers 
and  manpower  planners  with  this  circular  process  providing 
a  better  framework  for  the  system  designers. 
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CHAPTER  5 


QUALITY  ASSURANCE — MINIMIZING  THE  GAP  BETWEEN 
DESIGN  RELIABILITY  AND  FIELD  RELIABILITY 


The  reliability  intended  to  be  designed  into  a  system  is 
often  not  being  achieved  in  the  field.  It  is  the  task  of 
quality  assurance  to  help  minimize  this  gap.  We  found  that 
there  is  a  lack  of  adequate  guidance  for  program  managers 
on  how  to  influence  design  from  a  quality  assurance  stand¬ 
point.  Also,  files  on  contractors'  quality  histories  have 
not  been  fully  established  and  the  cross-service  product 
quality  deficiency  reporting  requirement  has  not  been  fully 
implemented.  Also,  there  is  a  shortage  of  engineers  in 
the  Government  who  are  qualified  to  address  quality  assurance. 

THERE  IS  A  GAP  BETWEEN  DESIGN 
RELIABILITY  AND  FIELD  RELIABILITY 

■  ■  ■  ■  ■  I  . .  II  »  II,  ■  I  I  .  ■  I.  »  »  !■  - - -  t 

DOD  has  been  placing  greater  emphasis  on  reliability  in 
recent  years,  particularly  in  the  late  1970s.  This  emphasis 
is  reflected  first  in  policy  revisions  including  the  new  DOD 
Acquisition  Directive  5000.40,  "Reliability  and  Maintainabil¬ 
ity,"  issued  July  8,  1980.  Not  only  has  there  been  greater 
policy  emphasis,  but  implementation  of  these  policy  changes 
can  be  seen  in  the  strengthening  of  contract  requirements, 
testing  procedures,  and  program  reviews. 

Some  of  the  hard  earned  reliability  improvements  in 
the  design  are,  however,  being  lost  in  the  manufacturing 
process  or  in  the  operating  environment  resulting  in  a  gap 
between  the  design  reliability  and  the  field  reliability. 

An  example  of  this  is  150,000  projectile  fuses  purchased 
by  the  Navy.  These  fuses  were  developed  to  obtain  commonality 
with  fuses  used  by  North  Atlantic  Treaty  Organization  nations. 
The  fleet  was  unable  to  use  these  fuses  because  the  tolerance 
levels  were  such  that  the  fuse  would  not  fire  from  a  gun 
that  was  not  perfectly  maintained,  a  near  impossible  task 
in  a  shipboard  environment. 

While  DOD  recognizes  that  this  gap  exists,  the  extent 
of  the  problem  is  very  difficult  to  quantify,  particularly 
in  the  complex/sophisticated  weaponry  of  today.  Correction 
of  the  problem,  however,  must  start  at  the  very  beginning 
of  the  acquisition  process  with  the  design  of  the  system. 
Because  of  the  importance  of  quality  in  system  designs,  we 
looked  at  DOD's  efforts  in  the  area  of  quality  assurance. 

f 
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CRITICALITY  OF  QUALITY  ASSURANCE  , 


It  is  the  task  of  quality  assurance  to  help  minimize 
the  gap  between  the  reliability  designed  into  a  system  (in¬ 
herent  reliability)  and  the  reliability  experienced  when  the 
system  is  deployed  (achieved  reliability).  DOD  defines  qual¬ 
ity  assurance  as: 

"A  planned  and  systematic  pattern  of  all  actions 
necessary  to  provide  adequate  confidence  that 
adequate  technical  requirements  are  established; 
products  and  services  conform  to  established 
technical  requirements;  and  satisfactory  perform¬ 
ance  is  achieved." 

With  the  technological  advances  of  recent  years,  the 
criticality  of  quality  assurance  has  increased  significantly. 
Sophisticated  techniques  are  being  used  to  test  the  quality 
and  reliability  of  components  early  in  the  system's  develop¬ 
ment.  'With  DOD's  increasing  emphasis  on  problem  prevention 
instead  of  detection,  quality  now  must  become  involved  in 
the  early  phases  of  the  acquisition  cycle. 

Quality  assurance  provides  confidence  that,  for  example: 

— The  tolerances  called  for  in  machining  are  cost 
effective . 

— The  material  to  be  used  is  compatible  with  the  manu¬ 
facturing  process. 

— The  parts  to  be  manufactured  from  the  drawings  can 
be  tested  to  confirm  compliance  with  the  design. 

Recent  designs  of  helicopter  main  rotor  blades  further 
illustrate  the  criticality  of  quality  assurance.  The  designs 
for  these  blades  have  used  composite  material  bonded  to  a 
metal  spar  under  heat  and  pressure. 
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FIGURE  4 

COMPOSITE  HELICOPTER  ROTOR  BLADE  CONSTRUCTION 


In  this  particular  illustration  two  separate  pieces  of 
composite  material,  (A)  and  (b),  are  to  be  bonded  to  metal 
spar  (C)  and  to  each  other.  The  critical  test  is  examination 
of  the  bond  seam  (D)  to  ensure  there  are  no  voids  where  the 
blade  might  begin  to  come  apart  under  the  high  stresses 
imposed  on  helicopter  blades. 

When  the  X-ray  examination  was  attempted,  it  was  dis¬ 
covered  that  the  bonding  material  specified  in  the  design 
was  of  such  a  composition  that  X-ray  photography  would  not 
work.  Since  no  alternative  type  of  testing  was  available, 
the  design  had  to  be  changed  to  specify  a  different  bonding 
material  that  could  be  X-rayed. 

In  this  illustration,  the  importance  of  adequate  quality 
assurance  testing,  and  therefore  of  testability,  is  clearly 
demonstrated.  The  significance  (or  criticality)  of  the 
required  design  change  is  a  matter  of  timing.  The  more 
complete  a  system  assembly  becomes  before  a  subsystem  or  com¬ 
ponent  is  found  to  be  defective  or  untestable,  the  more 
expensive  the  corrective  action  or  replacement  will  be. 

DIFFICULTIES  IN  ADDRESSING 
QUALITY  ASSURANCE 


When  a  manager  does  wish  to  address  quality  assurance, 
he  faces  several  problems.  First,  efforts  on  the  front  end 
of  a  design  process  to  address  quality  assurance  cost  money. 
Also,  although  failure  to  expend  the  effort  may  result  in 
lower  reliability  and  increased  support  costs,  the  benefits 
are  difficult  to  quantify.  The  savings  do  not  manifest  them¬ 
selves  in  acquisition  dollars  but  in  support  dollars.  Second, 
there  is  a  lack  of  role  and  responsibility  definition  for 
design  evaluation  from  the  quality  assurance  viewpoint. 

Third,  the  program  manager  may  lack  confidence  in  the  quality 
assurance  staff  due  to  a  scarcity  of  sufficiently  qualified 
personnel,  especially  engineers. 
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Lack  of  design  evaluation  guidance  and 
incomplete  policy  implementation 


While  the  DOD  Directive  4155.1  (Quality  Program),  revised 
in  1978,  recognizes  the  importance  of  quality  assurance,  OSD 
has  not  yet  taken  certain  steps  which  are  critical  to  suc¬ 
cessful  implementation  of  the  policy.  Part  of  the  problem 
may  be  a  serious  staffing  shortfall  in  the  quality  assurance 
function  at  OSD  where  only  one  person  is  devoted  to  the  policy 
aspects  of  quality  assurance.  This  individual  is  understand¬ 
ably  limited  in  how  much  he  can  accomplish.  Quality  assurance 
concerns  arising  during  his  absence  go  unaddressed.  Efforts 
to  unify  regulations  and  to  provide  detailed  guidance  to 
the  services  are  hampered.  For  example,  regarding  implementa¬ 
tion  of  the  revised  directive; 


— Directive  4155.1  recognizes  the  need  to  influence 

system  designs  by  requiring  specifications  in  contracts 
of  the  required  quality  characteristics.  OSD  has 
given  the  program  manager  the  responsibility  for 
reviewing  contractors'  designs  for  quality  assurance 
but  has  not  provided  guidance  on  how  this  is  to  be 
done . 


— The  quality  history  files  on  contractors,  required  in 
Directive  4155.1,  have  not  been  fully  established. 
Files  on  some  contractors  still  have  not  been  set 
up.  This  system  is  intended  to  prevent  the  awarding 
of  contracts  to  contractors  with  poor  quality  perform¬ 
ance  records. 


— The  cross-service  product  quality  deficiency  reporting, 
required  in  Directive  4155.1,  has  not  been  fully 
automated.  The  Army  and  the  Air  Force  are  the  only 
two  DOD  components  to  have  accomplished  this.  Also, 
a  central  data  bank  for  consolidating  the  DOD  component 
reports  into  one  report  has  not  been  set  up.  This 
would  greatly  facilitate  the  transfer  of  information 
on  quality  problems  among  all  DOD  components. 

Regarding  implementation  of  the  new  policy  emphasis  on 
quality  assurance  in  the  design  process,  we  examined  the 
standard  quality  specification  which  is  mandatory  for  use 
by  DOD  agencies  in  all  major  acquisition  contracts — MIL-Q- 
9858A,  "Quality  Program  Requirements."  We  found  it  to  be 
old  and  in  need  of  revision.  Originally  promulgated  in  1959, 
it  was  revised  and  reissued  in  1963.  It  has  not  been  signifi¬ 
cantly  changed  since.  The  primary  concern  of  the  document 
is  quality  control:  the  in-process  inspection  of  work  and 
correction  of  problems  discovered  during  production.  Design 


is  addressed  as  an  area  of  consideration  in  the  summary 
of  the  specification  but  is  not  referred  to  again,  except 
as  a  matter  for  corrective  (rather  than  preventive)  action. 
This  clearly  does  not  recognize  changes  toward  designed-in 
quality  which  have  occurred  in  the  acquisition  process  and 
the  quality  discipline  since  1963. 

In  recognition  of  deficiencies  in  the  specification, 

OSD  has  proposed  adoption  of  a  draft  revision  to  a  North 
Atlantic  Treaty  Organization  quality  publication.  The 
Organization's  quality  assurance  procedure  repeatedly  empha¬ 
sizes  the  need  to  aggressively  control  design  and  development. 

On  the  positive  side,  we  found  that  OSD  has  established 
a  standardization  project  run  by  the  Army.  The  objectives 
of  the  project  are  to  minimize  the  differences  among  service- 
level  implementation  policies  and  regulations  and  to  unify 
DOD's  basic  approach  to  quality  assurance. 

Need  for  additional  and  better 
qualified  staffing 

Dealing  with  today's  advanced  weapon  systems  is  dif¬ 
ficult  in  all  aspects,  including  the  attainment  of  adequate 
quality  assurance  during  the  acquisition  cycle.  In  the 
second  DOD  Conference  on  Quality  Assurance  Management  in 
March  1980,  the  introductory  remarks  include  this  comment; 

"The  increasing  complexity  of  our  weapon  systems 
and  equipment  has  caused  a  corresponding  complex¬ 
ity  in  the  tasks  facing  quality  assurance  person¬ 
nel." 

The  majority  of  the  DOD  quality  assurance  work  force 
has  had  little  training  in  quantitative  analysis  or  design 
disciplines  and  often  lacks  the  technical  expertise  to 
deal  with  complex  systems.  Also,  this  work  force  has  de¬ 
creased  45  percent  over  the  last  10  years  while  increased 
complexity  and  number  of  contracts  has  driven  the  workload 
requirements  up.  Pay  comparability  has  been  cited  as  a 
cause  for  much  of  the  turnover  problem  in  both  recruiting 
and  retention. 

RECENT  EMPHASIS  ON 
QUALITY  ASSURANCE 


There  is  an  increasing  awareness  at  the  higher  levels 
of  DOD  and  the  services  that  quality  is  a  problem  and 
that  many  changes  must  take  place  if  better  quality 
systems  are  to  be  produced.  We  noted  earlier  that  some 


actions  are  being  taken  such  as  the  Army's  quality  assurance 
standardization  project.  (See  p.  39.)  Following  are 
examples  of  others. 

Actions  have  been  initiated  to  help  alleviate  quality 
assurance  staffing  problems.  These  actions  include  the 
following . 

— The  DOD  Quality  and  Reliability  Assurance  Career  Board 
published  a  handbook  in  March  1980  with  information 
on  careers  in  the  field  to  attract  more  people. 

— DOD  is  also  working  with  the  Office  of  Personnel 
Management  to  review  the  current  job  classification 
standards  in  the  quality  assurance  field.  For  example, 
the  Navy  is  promoting  the  establishment  of  a  new 
engineering  classification,  the  quality  assurance 
engineer . 

The  move  to  upgrade  job  descriptions  and  corresponding 
pay  levels,  however,  is  recent  and  is  expected  to  take  2 
years  to  complete.  In  the  meantime,  internal  training 
programs  at  the  service  levels  are  being  relied  on  to  improve 
the  quality  assurance  staff. 

— The  Army  has  three  intern  programs  for  the  quality 
assurance  work  force  and  has  recently  expanded  the 
engineer  program  from  18  weeks  to  1  year  of  classroom 
training . 

— The  Navy  has  a  3-year  old  quality  engineer  intern 
program  with  most  of  the  training  accomplished  by 
contractors  in  accordance  with  Navy  requirements 
and  specifications. 

— The  Air  Force  is  developing  an  intern  program  for 
acquisition  quality  assurance  personnel  which 
should  be  underway  shortly. 

Regarding  the  need  to  justify  the  expenditures  necessary 
for  adequate  "front-end"  quality  assurance,  one  service 
plans  to  develop  an  itemized  checklist  to  allow  a  program 
manager  to  identify  the  cost  of  successively  more  intensive 
levels  of  attention  to  quality  assurance  during  the  acquisi¬ 
tion  cycle. 

NEED  FOR  NEW  TERMINOLOGY 

There  is  general  confusion  as  to  the  meaning  of  quality, 
quality  control,  and  quality  assurance.  We  suggest  a 
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phrase  of  increasing  popularity,  “product  assurance,"  as  the 
beginning  of  the  solution.  Product  assurance  has  been  defined 
as  follows: 


"PRODUCT  ASSURANCE  is  the  application  of  inter¬ 
disciplinary  skills  to  accomplish  the  preventive 
and  conformance  activities  necessary  to  assure: 
that  requirements  are  properly  specified,  that 
the  design  will  achieve  these  requirements  and 
that  the  ultimate  product  and/or  services  will 
perform  their  intended  functions  in  the  opera¬ 
tional  environment  for  the  period  specified." 

This  quote,  taken  from  the  Air  Force  Systems  Command's 
1979  "Quality  Horizons  Final  Report,"  sums  up  the  concept  of 
an  interdisciplinary  approach  to  quality  which  now  is  in  use, 
in  part,  by  most  of  the  services.  Product  assurance  goes 
beyond  quality  control  and  beyond  quality  assurance  to  incor¬ 
porate  reliability,  maintainability,  and  other  related 
disciplines . 

CONCLUSIONS 


Increasing  complexity/ sophistication  of  weapon  systems 
has  already  led  to  quality  considerations  going  beyond 
traditional  concerns  with  quality  control  into  participation 
in  the  transition  from  concept  to  design.  While  the 
importance  of  quality  assurance  in  the  design  process  is 
becoming  well  recognized,  the  influence  quality  assurance 
will  have  on  design  is  being  diluted  by  a  number  of  factors 
which  need  to  be  addressed  by  OSD. 

First,  the  focal  point  of  all  program  efforts — the 
program  manager — has  been  given  much  direction  about 
quality,  but  little  help  or  incentive  in  pursuing  it.  He 
has  been  directed  to  evaluate  designs  for  quality  and  is  now 
in  need  of  comprehensive  guidance  on  how  this  is  to  be  done. 
The  principal  quality  assurance  specification  does  not  meet 
this  need. 

Second,  the  quality  history  files  on  contractors  have 
not  been  fully  established  and  the  cross-service  product 
quality  deficiency  reporting  requirement  has  not  been  fully 
implemented.  Until  these  actions  take  place,  selection  of 
contractor (s)  to  develop  a  new  weapon  system  cannot  adequately 
consider  contractors'  quality  histories. 

Finally,  there  is  a  shortage  in  DOD  of  engineers  ade¬ 
quately  trained  in  quality  assurance.  Actions  are  needed 
to  address  this  situation  and  strengthen  the  quality  assurance 
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work  force.  Some  changes  are  being  made,  but  they  do  not 
appear  to  be  sufficient  considering  the  needs. 


RECOMMENDATIONS  TO  THE 
SECRETARY  OF  DEFENSE 

We  recommend  that  the  Secretary  of  Defense 

—produce  comprehensive  guidance  as  to  how  designs  should 
be  evaluated  for  quality  assurance, 

— ensure  that  the  quality  history  files  on  contractor 
are  fully  established  and  that  the  cross-service 
product  quality  deficiency  reporting  requirement  is 
fully  implemented ,  and 

— strengthen  the  quality  assurance  work  force  so  as  to 
permit  their  active  involvement  in  the  design  phase 
of  the  acquisition  process. 

AGENCY  COMMENTS 

DOD  agrees  with  our  findings,  conclusions,  and  recom¬ 
mendations.  (See  app.  III.) 
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LOCATIONS  VISITED  DURING  REVIEW 


UNITED  STATES 
OSD 


— Assistant  Secretary  of  Defense  (Manpower,  Reserve 
Affairs,  &  Logistics),  Washington,  D.C. 

— Under  Secretary  of  Defense  for  Research  &  Engineer¬ 
ing,  Washington,  D.C. 


— Office  of  the 

Army 

— Headquarters , 
Georgia. 

— Headquarters , 
and  Readiness 

— Headquarters , 

— Headquarters , 
Command ,  Fort 


Joint  Chiefs  of  Staff, 


Army  Forces  Command, 


Washington,  D.C. 


Fort  McPherson, 


United  States  Army  Materiel  Development 
Command,  Alexandria,  Virginia. 

United  States  Army,  Washington,  D.C. 

United  States  Army  Training  and  Doctrine 
Monroe,  Virginia. 


— United  States  Army  Materiel  Readiness  Support  Activity, 
Lexington,  Kentucky. 


— United  States  Army  Human  Engineering  Laboratory, 
Aberdeen  Proving  Grounds,  Maryland. 


— United  States  Army  Materiel  Systems  Analysis  Agency, 
Aberdeen  Proving  Grounds,  Maryland. 


— Headquarters,  4th  Infantry  Division,  Fort  Carson, 
Colorado . 


— United  States  Army  Tank  Automotive  Materiel  Readiness 
Command,  Warren,  Missouri. 

— United  States  Army  Missile  Command,  Huntsville, 
Alabeuna . 


— United  States  Army  Infantry  Center,  Fort  Benning, 
Georgia . 

— Headquarters,  United  States  Army  Electronics  Research 
and  Development  Command,  Adelphi,  Maryland. 
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— United  States  Army  Troop  Support  and  Aviation  Materiel 
Readiness  Command,  St.  Louis,  Missouri. 


Navy 


— Office  of  the  Chief  of  Naval  Operations,  Washington, 

D.C. 


— Headquarters,  Naval  Material  Command,  Washington, 

D.C. 


— Naval  Air  Systems  Command,  Washington,  D.C. 

— Naval  Electronics  Systems  Command,  Washington,  D.C. 

— Naval  Sea  Systems  Command,  Washington,  D.C. 

— Naval  Supply  Systems  Command,  Washington,  D.C. 

— Headquarters,  Commander-in-Chief ,  U.S.  Atlantic  Fleet, 
Norfolk,  Virginia. 

— U.S.  Naval  Surface  Forces,  Atlantic  Fleet,  Norfolk, 
Virginia . 

— U.S.  Naval  Air  Forces,  Atlantic  Fleet,  Norfolk, 
Virginia . 

— U.S.  Naval  Submarine  Forces,  Atlantic  Fleet,  Norfolk, 
Virg inia . 

— Commander  Operational  Test  and  Evaluation  Forces, 
Norfolk,  Virginia. 

— U.S.  Naval  Ship  Engineering  Center,  Norfolk  Division, 
Norfolk,  Virginia. 

— U.S.  Navy  Fleet  Analysis  Center,  Corona,  California. 

— U.S.  Marine  Corps  Headquarters,  Deputy  Chief  of 
Staff  for  Installations  and  Logistics,  Arlington, 
Virginia . 

Air  Force 


— Headquarters,  United  States  Air  Force,  Washington, 

D.C. 
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— Air  Force  Logistics  Command,  Wr ight-Patterson  Air  Force 
Base,  Ohio. 

— Air  Force  Systems  Command,  Andrews  Air  Force  Base, 
Maryland . 

— Air  Force  Institute  of  Technology,  Wright-Patterson 
Air  Force  Base,  Ohio. 

—Headquarters,  Air  Force  Aeronautical  Systems  Division, 
Wright-Patterson  Air  Force  Base,  Ohio. 

— Headquarters,  Air  Force  Tactical  Air  Command,  Langley 
Air  Force  Base,  Virginia. 

— Air  Force  Aerospace  Medical  Research  Laboratory, 
Wright-Patterson  Air  Force  Base,  Ohio. 

— Air  Force  Flight  Test  Center,  Edwards  Air  Force  Base, 
California . 

Contractors 

—McDonnell  Aircraft  Company,  St.  Louis,  Missouri. 

— General  Dynamics,  Fort  Worth,  Texas;  Pomona,  Cali¬ 
fornia  . 

— Bendix  Corporation,  Teterboro,  New  Jersey. 

— Bell  Helicopter  Textron,  Fort  Worth,  Texas. 

— IBM,  Federal  Systems  Division,  Manassas,  Virginia. 

— Hughes  Aircraft  Company,  Culver  City,  California. 

— Chrysler  corporation,  Warren,  Michigan. 

Other  U.S.  locations 

— CACI,  Inc.,  Arlington,  Virginia. 

— Defense  Audit  Service,  Washington,  D.C. 

—Defense  Logistics  Agency,  Washington,  D.C. 

— Defense  Systems  Management  College,  Fort  Belvoir, 
Virginia . 

—Society  of  Logistics  Engineers,  Washington,  D.C. 
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EUROPE 

— Headquarters,  U.S.  European  Command,  Stuttgart, 

Germany . 

Army 

— Headquarters,  U.S.  Army  Europe,  Heidelberg,  Germany. 

— Headquarters,  U.S.  Army  VIII  Corps,  Stuttgart,  Germany. 
— Headquarters,  3rd  Infantry  Division,  Wurzburg,  Germany. 

1.  Headquarters,  3rd  Infantry  Division  Artillery, 
Wurzburg,  Germany. 

2.  2nd  Infantry  Brigade,  Hohenfelds,  Germany. 

1st  Battalion,  10th  Artillery,  Scheinfurt,  Germany. 
3rd  Squadron,  7th  Cavalry  Regiment,  Scheinfurt, 
Germany . 

3.  3rd  Brigade  A  Schaffenburg,  Germany. 

703rd  Direct  Support  Maintenance  Battalion, 
Kitzingen,  Germany. 

3rd  Aviation  Battalion,  Kitzingen,  Germany 

— Headquarters,  17th  Field  Artillery  Brigade,  VII  Corps 
Artillery,  Augsburg,  Germany. 

1.  1st  Battalion,  18th  Artillery,  Augsburg,  Germany. 

2.  1st  Battalion,  36th  Artillery,  Augsburg,  Germany. 

3.  1st  Battalion,  30th  Artillery,  Augsburg,  Germany. 

— Headquarters,  1st  Armored  Division,  Ansbach,  Germany. 

1.  3rd  Brigade,  Bamberg,  Germany. 

2.  501st  Aviation  Battalion,  Ansbach,  Germany. 

3.  Division  Support  Command,  Ansbach,  Germany. 

4.  1st  Battalion,  94th  Artillery,  Grafenwohr,  Germany. 

5.  1st  Battalion,  37th  Armor,  Ansbach,  Germany. 

6.  Headquarters,  Division  Artillery,  Ansbach,  Germany. 
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Navy 


--Headquarters,  U.S.  Naval  Forces  Europe,  London, 
England. 

— Headquarters,  U.S.  Sixth  Fleet,  Gaeta,  Italy. 

—Commander,  U.S.  Sixth  Fleet  Battle  Forces,  aboard 
U.S.S.  Independence  (CVA-62),  Naples,  Italy. 

— Headquarters,  U.S.  Sixth  Fleet  Maritime  Surveillance 
and  Reconnaissance  Force,  Naples,  Italy. 

— Headquarters,  U.S.  Sixth  Fleet  Attack  Submarine  Force, 
Naples,  Italy. 

— Officers  and  crew  members  of  U.S.S.  Independence 
(CVA-62),  U.S.S.  Mahan  (DDG-42),  and  U.S.S.  Memphis 
(SSN-691),  Naples,  Italy. 

Air  Force 


— Headquarters,  U.S.  Air  Force  Europe,  Ramstein,  Germany. 
—Headquarters,  17th  Air  Force,  Sembach,  Germany. 

1.  The  86th  Tactical  Fighter  Wing,  Ramstein,  Germany. 

2.  The  36th  Tactical  Fighter  Wing,  Bitburg,  Germany. 

3.  A- 10  Unit,  Sembach,  Germany. 


47 


APPENDIX  II 


APPENDIX  II 


OTHER  RELEVANT  GAO  REPORTS 


ISSUED  BETWEEN  JANUARY  1979  AND  NOVEMBER  1980  1 / 


Date 

Number 

Title 

11/13/80 

C-PSAD-81-4 

Improvements  in  Performance  and 
Reliability  Should  Govern  Future 
Procurement  of  Army’s  Copperhead 
Projectile  (Classified) 

10/24/80 

C-LCD-81-1 

Navy's  Antisubmarine  Warfare 
Capability — Is  It  Sufficient? 
(Classified) 

9/30/80 

PSAD-80-72 

Are  Management  Problems  in  the 
Acquisition  of  Aircraft  Gas 
Turbine  Engines  Being  Corrected? 

9/16/80 

LCD-80-102 

Survey  of  the  Readiness  of  Minute- 
men  Missiles 

9/4/80 

LCD-80-106 

Survey  of  DOD’s  Management  of 
Automatic  and  General  Purpose 
Electronic  Test  Equipment 

8/21/80 

LCD-80-78 

Opportunities  for  Future  Improve¬ 
ment  of  Government  Logistics 
Management 

8/20/80 

LCD-80-89 

F-16  Integrated  Logistic  Support: 
Still  Time  to  Consider  Alterna¬ 
tives 

7/22/80 

FPCD-80-58 

Actions  to  Improve  Parts  of  the 
Military  Manpower  Mobilization 
System  Are  Underway 

6/30/80 

PSAD-80-61 

Implications  of  Highly  Sophisti¬ 
cated  Weapon  Systems  on  Military 
Capabilities 

l/lnstructions  for  obtaining  copies  of  our  reports  are  on 
the  Inside  front  cover  of  this  report.  To  obtain  copies 
of  classified  reports,  security  clearance  information  must 
be  provided  along  with  a  demonstrated  need-to-know. 
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Date 

6/12/80 

6/6/80 

5/9/80 

4/14/80 

4/1/80 

2/20/80 

2/15/80 

2/7/80 

1/29/80 

12/11/79 

12/11/79 

11/26/79 


Number 

PSAD-80-43 

LCD-80-65 

PSAD-80-40 

C-FPCD-80-3 

LCD-80-48 

PPCD-80-10 

PPCD-80-31 

LCD-80-30 

PSAD-80-20 

LCD-80 -2 

FPCD-80-6 

C-FPCD-80-1 


Title 

Issues  Identified  in  21  Recently 
Published  Major  Weapon  System 
Reports 

Operational  and  Support  Costs  of 
the  Navy's  P/A-18  Can  Be  Sub¬ 
stantially  Reduced 

Is  the  Joint  Air  Force/Navy 

Alternate  Engine  Program  Work¬ 
able?  GAO  Thinks  Not  as  Pre¬ 
sently  Structured 

Overview  of  the  Manpower  Effec¬ 
tiveness  of  the  All-Volunteer 
Force  (Classified) 

Logistics  Management  Issues  Staff 
Study 

Attrition  in  the  Military — An 
Issue  Needing  Management 
Attention 

The  Navy’s  Pilot  Shortage:  A 
Selective  Bonus  and  Other 
Actions  Could  Improve  Retention 

Increased  Standardization  Would 
Reduce  Costs  of  Ground  Support 
Equipment  for  Military  Aircraft 

XM-1  Tank's  Reliability  is  Still 
Uncertain 

Improving  the  Effectiveness  of 
Joint  Military  Exercises — An 
Important  Tool  for  Military 
Readiness 

Estimates  of  Available  Hours  for 
Military  Personnel  in  Wartime 
Distort  Force  Requirements  and 
Planning 

Active  Duty  Manpower  Problems — 
Barrier  to  Mission  Accomplish¬ 
ment  (Classified) 
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Date 

Number 

Title 

11/13/79 

C-PSAD-80-2 

Army  Operational  Test  and  Evalua¬ 
tion  Needs  Improvement  (Clas¬ 
sified) 

11/8/79 

PSAD-80-6 

Impediments  to  Reducing  the  Costs 
of  Weapon  Systems 

11/6/79 

LCD-80- 11 

Modernizing  the  Air  Reserve 

Force — More  Emphasis  on  Logis¬ 
tics  Support  Needed 

10/30/79 

C-LCD-80-2 

A-10  Aircraft  Logistics  Support 

Can  Be  Better  Matched  with 
Operational  Requirements 
(Classified) 

10/12/79 

LCD-80-5 

DOD's  Material  Readiness  Report 
to  the  Congress — Improvements 
Needed  to  Better  Show  the  Link 
Between  Funding  and  Readiness 

9/28/79 

LCD-79-414 

Alternatives  to  Consider  in  Plan¬ 
ning  Integrated  Logistics  Sup¬ 
port  for  the  Trident  Submarine 

9/11/79 

PSAD-79-99 

Manufacturing  Technology — A  Cost 
Reduction  Tool  at  the  Department 
of  Defense  that  Needs  Sharpening 

8/20/79 

LCD-79-423 

Letter  Report  on  GAO's  Concerns 
with  the  Readiness  of  U.S. 

Forces  (Classified) 

8/9/79 

PSAD-79-95 

Army  Procurement  of  lOkW,  60Hz 

Gas  Turbine  Generators  is  Highly 
Questionable 

1/21119 

FPCD-79-13 

DOD  Oversight  of  Individual  Skill 
Training  in  the  Military  Serv¬ 
ices  Should  be  More  Comprehen¬ 
sive 

7/11/79 

FPCD-79-58 

Critical  Manpower  Problems  Re¬ 
strict  the  Use  of  National 

Guard  and  Reserve  Forces 

6/28/79 

FPCD-79-3 

Can  the  Individual  Reserves  Fill 
Mobilization  Needs? 
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Date 

Number 

Title 

6/25/79 

PSAD-79-86 

Effectiveness  of  DOD's  Development 
Test  and  Evaluation 

s/nn9 

FPCD-79-40 

Problems  in  Getting  People  Into 
the  Active  Forces  After  Mobili¬ 
zation 

5/10/79 

LCD-79-407 

If  Army  Helicopter  Maintenance  Is 
to  Be  Ready  for  Wartime,  It  Must 
Be  Made  Efficient  and  Effective 
in  Peacetime 

4/25/79 

PSAD-79-64 

Digests  of  Major  Weapon  System 
Reports  Issued  January  and 
February  1979 

4/25/79 

LCD-79-404 

Can  the  Army  and  Air  Force  Re¬ 
serves  Support  the  Active  Force 
Effectively? 

4/23/79 

LCD-79-406 

The  United  States  Air  Force  Tac¬ 
tical  Air  Command — Is  It  Ready? 
Can  It  Fulfill  U.S.  Commitments 
to  Rapidly  Increase  Its  Forces 
in  Europe? 

4/28/79 

PSAD-79-44 

The  Effectiveness  of  the  F-14A/ 
Phoenix  Weapon  System  Is  Mar¬ 
ginal  at  Best  Against  the  Cur¬ 
rent  and  Postulated  Threat 
(Classified) 

2/20/79 

PSAD-79-9 

Observations  on  Office  of  Manage¬ 
ment  and  Budget  Circular  A-109 — 
Major  System  Acquisitions  by 
the  Department  of  Defense 

2/12/79 

LCD-78-430 

Readiness  of  Conventional  U.S. 

Air  Forces  in  Europe — Selected 
Aspects  and  Issues 

2/6/79 

LCD-78-417A 

Marine  Amphibious  Forces t  A 

Look  at  Their  Readiness,  Role, 
and  Mission 

1/24/79 

FPCD-78-82 

DOD's  "Total  Force  Management"— 
Fact  or  Rhetoric? 
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RESEARCH  AND 
ENGINEERING 


THE  UNDER  SECRETARY  OF  DEFENSE 
WASHINGTON  DC  20MI 


f  .>r  ,^  ■.  /'■?  ^ 


(t1  DEC  1880 


Mr.  W.H.  Sheley,  Jr. 

Acting  Director,  Procurement  and  Systems 
Acquisition  Division 
U.S.  General  Accounting  Office 
Washington,  D.C.  20546 


Dear  Mr.  Sheley; 

This  is  in  reply  to  your  letter  to  the  Secretary  of  Defense  regarding  your 
draft  report  on  "Ownership  Considerations  Need  More  Eaqihasis  in  Weapon  Systems 
Design",  OSD  Case  #5538,  GAO  Code  951516. 

We  have  reviewed  the  draft  report  and  believe  it  is  excellent  and  well-balanced. 
DoD  agrees  with  GAO  in  the  need  to  strengthen  these  activities.  The  report 
accurately  represents  our  recent  efforts  to  increase  the  emphasis  on  ownership 
and  support  considerations  during  the  development  of  DoD  systems.  The  report 
discusses  the  degree  of  difficulty  associated  with  many  of  the  areas  of  logis¬ 
tics  analysis  and  projection  as  well  as  the  difficulty  in  testing  a  system's 
logistics  supportability .  The  report  also  recognises  recent  policies  issued 
by  DoD  to  emphasize  these  very  important  considerations  and  the  fact  that  there 
has  not  been  sufficient  time  to  accrue  full  benefit  from  these  recent  policy 
changes.  The  OoO  has  initiated  many  activities  to  follow-up  on  the  new  logistic 
support  policies. 

The  attached  comments  have  been  infonsally  discussed  with  your  staff.  They 
serve  to  clarify  soaw  of  the  significant  observations  and  recommendations  in 
the  report.  We  would  appreciate  your  incorporation  of  these  views  in  the  final 
report. 

We  appreciate  the  opportunity  to  consent  on  the  draft  report  and  are  willing 
to  discuss  further  with  the  GAO  staff  any  of  the  points  contained  herein. 


Attachment 

a/s 


Sincerely, 


Wiftw  B.  LaB^r^ 
principal  Depi'ty 
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COMMENTS  OH  OSD  #5538 

GAO  Draft  Report  "Ownership  Considerations  Heed  More  Emphasis  in  Weapon  System 
Design" 


GAO  Recommendation:  "Develop  new  or  modify  current  maintenance  data  reporting 
procedures  to  provide  the  information  needed  for  performing  logistic  support 
analysis." 

OSD  Responses:  Concur  in  principle. 

There  is  a  need  for  additional  and  better  historical  data  to  support  the 
performance  of  logistic  support  analysis.  It  is  not  clear  that  in  all  cases 
this  data  should  be  drawn  from  a  maintenance  data  system.  For  example,  the 
Air  Force  is  addressing  this  specific  problem  with  a  project  entitled  "Product 
Performance  Feedback  System.”  This  project  deals  with  analysis  results  rather 
than  data  products.  Across  the  wide  scope  of  DoD  missions  and  systems  there 
are  a  number  of  different  solutions  which  will  address  this  specific  problem. 

In  some  instances  a  new  or  isq>roved  maintenance  data  system  may  be  the  answer, 
in  other  situations  other  alternatives  may  be  preferable.  The  new  Department 
of  Defense  Directive  on  reliability  and  maintainability  (DoDD  5000.40)  requires 
DoD  in-service  data  collection  systems  to  report  the  measured  values  of  system 
R&H  parameters  which  relate  to  readiness,  SMintenance  manpower  and  logistics 
support  cost.  While  each  Service's  ia^>lementation  may  be  different,  the  need 
for  this  data  has  been  recognized. 

GAO  Recommendation:  "Establish  logistic  support  research  and  study  programs 
to  develop  improved  quantitative  methods  for  assessing  (1)  the  adequacy  of 
logistic  support  plans,  resources,  and  support  related  parameters,  and  (2)  the 
benefits  of  design  changes  in  the  acquisition  process  to  improve  the  future 
logistic  supportability  of  a  system." 

OSD  Responses:  Concur. 

On  September  3,  1980  direction  was  sent  from  OSD  to  the  Secretaries  of  the  Mili¬ 
tary  Departments  to  initiate  a  Research  and  Development  program  for  improvement 
of  weapon  support.  The  scope  of  this  program  will  include  logistic  concept 
development,  hardware  techniques  and  design  tradeoffs,  and  weapons  system  demons¬ 
tration  projects. 

GAO  Recommendation:  "Provide  guidance  to  insure  that  critical  program  docu- 
menta  (for  example,  misaion  element  needs  statements,  request  for  proposals 
and  contracts)  contain  the  language  needed  to  obtain  systems  which  are  logis- 
tically  supportable." 

OSD  Responses:  Concur. 

Both  DoD  Directive  5000.39  and  5000.40  both  provide  implesientation  guidance 
on  considerations  that  should  be  included  in  various  program  docuaieots  related 
to  support.  For  exanqile,  5000.39  requires  that  manpower  and  logistic  resource 
constraints  be  identified  in  the  HENS;  that  contract  requirements  for  full-scale 
development  clearly  define  baseline  operational  scenerioa,  a  baseline  maintenance 
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concept,  and  readiness  and  wartime  employment  objectives;  and  that  a  preliminary 
manning  document  and  supporting  analysis  be  available  by  the  production  decision 
point.  DoOO  5000.40  includes  the  requirement  to  specify,  in  the  conceptual 
phase,  the  system  life  profiles  and  tenative  goals  for  the  appropriate  system 
R&M,  and  the  requirement  to  translate  R&M  threshold  objectives  into  mimisum 
acceptable  values  in  contracts  for  both  contractor  and  governatent  furnished 
equipment  during  the  full-scale  development  phases.  We  will,  over  the  next 
several  years,  evaluate  the  application  of  this  guidance  to  determine  its 
strenths  and  weaknesses  and  will  issue  supplemental  guidance  as  appropriate. 

GAO  Recommendation:  "Provide  for  improved  testing  and  evaluation  of  the 
logistic  supportability  of  systems  before  they  are  deployed." 

OSD  Response:  Concur. 

As  discussed  in  the  draft  report,  OSD  and  the  Services  have  had  a  joint  study 
underway  since  March  1979  to  address  the  area  of  operational  suitability 
evaluation.  The  first  phase  of  this  study  examined  problems  involved  in  test 
and  evaluation  of  logistic  supportability.  The  phase  one  study  results  sum¬ 
marized  in  the  GAO  report  point  out  the  many  constraints  and  difficulties 
involved  in  implementing  existing  DoO  policies  requiring  operational  suitability 
evalution.  Guidelines  for  improvements  in  supportability  evaluation  are  being 
developed  in  the  second  phase  of  the  study. 

GAO  Recommendation:  "Revise  the  Logistic  Support  Analysis  Military  Standard 
to  emphasize  that  logistic  supportability  is  now  as  important  as  cost,  sche¬ 
dule  and  performance  and  must  receive  appropriate  emphasis  in  system  design." 

OSD  Response:  Concur  in  principle. 

The  Military  Standard  is  now  under  revision  to  bring  the  standard  into  agreement 
with  the  new  logistics  policies.  However,  the  Military  Standard  is  not  the 
best  document  to  provide  priorities  to  the  program  manager.  Other  policy 
directives,  including  DoDD  5000.1  and  DoDD  5000.2  as  recently  revised,  now 
contain  such  an  eoiphasis  statement.  Particular  emphasis  will  be  given  to  the 
developaient  of  logistic  analysis  guidelines  during  the  critical  concept  and 
advanced  development  phases,  where  system  level  tradeoffs  affecting  cost, 
schedule  and  performance  are  made.  We  believe  this  will  provide  the  appropriate 
impleaientation  guidelines. 

GAO  Recosmiendation:  "Modify  human  factor  specifications,  standards  and  hand¬ 
books  used  in  system  design  so  that  they  adequately  address  all  human  limita¬ 
tions  including  skill  levels,  proficiency,  availability,  environmental  stress 
and  fatigue  which  can  result  in  human-induced  system  failures." 

OSD  Response:  Concur. 

GAO  Recommendation:  "Develop  common  SKthodologies  and  data  sources  for  use  by 
system  designers  in  forecasting  skill  levels  of  military  personnel  5  to  10 
years  in  the  future." 
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OSD  Response:  Concur  in  principle. 

The  forecasting  of  skill  levels  is  necessary  as  an  input  to  the  system  defini¬ 
tion  effort,  however,  the  skill  level  is  the  result  of  many  things  including 
the  projected  training  and  the  training  that  is  required  as  a  result  of  system 
definition.  We  agree  that  this  circular  process  should  be  better  addressed 
and  provide  a  better  fraaiework  for  the  system  designers. 

GAO  Recommendation:  "Ensure  that  all  major  systems  are  subjected  to  adequate 
testing  and  examination  from  a  human  factors  standpoint." 

OSD  Responses:  Concur. 

This  recommendation  should  be  clarified.  The  use  of  the  word  testing  might 
infer  that  more  "after-the-fact"  emphasis  is  recommended.  A  preferred  solu¬ 
tion  is  to  provide  emphasis  at  all  stages  of  the  system's  development  from 
Initiation  to  conclusion. 

GAO  Recommendation:  "Produce  comprehensive  guidance  for  program  managers  as 
to  how  a  design  should  be  evaluated  for  quality  assurance.” 

OSD  Responses:  Concur. 

The  establishsKnt  of  this  guidance  is  only  one  part  of  the  solution  to  the 
identified  problem.  In  addition  to  guidance,  the  proper  management  emphasis 
is  also  required. 

GAO  Recommendation:  "Inqtlement  the  data  feedback  system  on  contractor  quality 
histories,  and  insure  that  the  cross-service  product  quality  deficiency  reporting 
requirement  is  carried  out." 

OSD  Response:  Concur. 

The  cross-service  reporting  system  was  established  in  1979,  but  all  services/ 
agencies  have  not  yet  implemented  the  program.  The  Army  has  completely  func¬ 
tioning  program  and  the  Air  Force  is  close  behind. 

GAO  Recommendation:  "Strengthen  the  quality  assurance  workforce  so  as  to 
permit  their  active  involvement  in  the  design  process." 

OSD  Response:  Concur. 

OSD  General  Comments: 


GAO  Observation  (pages  vi):  "Government  quality  assurance  engineers  are  in 
short  supply  and  lack  adequate  training." 

OSD  Comment;  Engineers  in  general  are  in  short  supply  and  engineers  working 
in  quality  assurance  are  no  exception.  These  are  qualified  engineers  who  work 
in  the  quality  assurance  area  but  who  may  need  BK>re  training  in  quality 
assurance  subjects . 


I 
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GAO  Observation  (page  32):  "The  MENS  statements  are  to  contain  logistics 
considerations,  but  only  the  Air  Force  has  issued  guidance..." 

OSD  Co— ent:  The  Army  published  a  letter  of  instruction  for  MENS  preparation 
on  January  7,  1980.  This  instruction  provides  detailed  guidance  for  preparation 
and  submission  of  MENS. 

GAO  Observation  (page  54):  "The  DoD  acquisition  directives  (5000  series), 
however  emphasize  repeatedly  that  the  concerns  of  the  manager  ought  to  be 
cost,  schedule,  performance,  and  more  recently,  supportability." 

OSD  Coawient:  This  observation  does  not  recognize  the  discussion  of  quality/ 
reliability  and  maintainability  contained  in  DoDI  5000.2  Specific  discussion 
is  in  paragraphs  C.8.2  and  C.9. 

GAO  Observation  (page  57):  ".  .  .on  how  to  review  designs  for  quality  assu¬ 

rance,  we  examined  the  standard  quality  specification  which  is  mandatory  for 
use  by  DoD  agencies  —  MIL-Q-9858A.  .  .” 

OSD  Comment:  Mil-Q-9858A  is  for  contractual  application  only,  it  does  not 
apply  to  "in-house"  DoD  activities.  The  document  is  currently  under  revision 
as  part  of  the  NATO  AQAP-1  activity. 

GAO  Observation  (page  61):  "We  suggest  a  phrase  of  increasing  popularity, 
"product  assurance,"  as  a  beginning  of  the  solution.  Product  assurance  has 
been  defined  as..." 

OSD  Coasnent:  The  definition  as  presented  in  the  draft  GAO  report  was  taken 
from  the  Air  Force  Systems  Command  1979  "Quality  Horizons  Final  Report."  It  is 
just  an  opinion  of  what  should  be  defined  as  the  integrated  subject  of  quality, 
quality  control,  quality  assurance,  reliability,  etc. 

GAO  Observation  (page  75):  "Examples  of  military  equipsient  reported  by  the  ser¬ 
vices  to  be  undependable  and  difficult  to  support  and  operate." 

OSD  Comment:  Details  on  these  problem  statements  should  be  changed. 

TOW  Problem;  Problem  statement  is  not  correct.  The  Army  is  procuring  a 
new  type  battery  and  enqploying  TOW  vehicle  power  conditioners,  thus  the  pro¬ 
blem  has  been  eliminated  duirng  the  past  18  months. 

DRAGON  Problem:  Although  there  have  been  some  component  malfunctiona 
(defective  thrusters)  these  problems  were  detected  prior  to  deployment  and  use, 
and  thus  cannot  be  said  to  be  said  to  be  contributing  to  misses. 

Track  Problem:  Change  to  read:  The  end  connectors  and  track  pads  for  this 
track  must  be  replaced  at  1500-2000  miles  and  the  entire  track  between  SOOO  and 
6000  miles.  The  track  cost  approximately  $23,000  per  set.  The  track  pads  can 
be  replaced  without  rebuilding  the  track  and  allows  the  extension  of  track  life 
described  above. 


56 


APPENDIX  IV 


APPENDIX  IV 


SUMMARY  OF  SPECIFIC  COMMENTS 
MADE  BY 

OUR  PROCUREMENT  ADVISORY  PANEL 

1.  One  of  the  serious  problems  affecting  ownership  consid¬ 
erations  in  the  acquisition  process  is  a  lack  of  continu¬ 
ity  in  program  management — there  is  a  new  program  manager 
(a  military  officer)  about  every  3  to  4  years.  The  pro¬ 
gram  manager  is  most  concerned  about  what  happens  on  his 
"watch"  and  thus  is  not  inclined  to  place  emphasis  on 
factors  such  as  supportability,  human  reliability,  and 
quality  assurance  where  the  benefits  are  realized  when 
the  system  is  deployed.  Perhaps  a  civilian  should  be 
designated  as  project  manager  and  kept  in  that  position 
until  the  system  is  deployed. 

2.  Contractors  have  the  "know-how"  to  design  more  support¬ 
able  systems.  If  the  Government  would  give  the  contrac¬ 
tor  the  financial  incentive  and  responsibility,  the 
system  designed  would  be  much  more  supportable  and  the 
financial  investment  will  be  returned  in  reduced  operat¬ 
ing  and  support  costs . 

3.  Contracts  need  to  contain  language  which  provide  the  con¬ 
tractor  with  qualitative  and  quantitative  requirements 
that  force  designers  not  to  ignore  supportability. 

4.  The  Government  must  encourage  the  application  of  new 
technology  to  the  design  of  the  weapon  system  support 
systems  (for  example,  test  and  diagnostic  equipment) 
to  improve  supportability. 

5.  The  logistic  support  analysis  does  not  necessarily 
have  to  be  complex  and  costly.  The  analysis  needs 
to  be  tailored  to  the  specific  needs  of  the  weapon 
system  development  program  using  simplified  models 
and  a  good  knowledge  of  design  alternatives  as  a 
substitute  for  complex  software. 

6.  There  are  serious  semantic  problems  in  the  whole  area  of 
support.  There  is  no  common  or  generally  agreed  upon 
set  of  terms  and  definitions  in  the  area  of  supportabil¬ 
ity.  DOD  should  revise  its  glossary. 

7.  There  are  too  many  directives,  specifications,  regula¬ 
tions,  and  so  forth.  A  major  effort  is  needed  to  reduce 
the  number. 
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8.  Much  attention  needs  to  be  given  to  how  designers 
can  design  for  supportability . 

9.  The  Office  of  Management  and  Budget's  Circular  A-109, 
approved  in  1976,  should  result  in  the  design  of  more 
supportable  systems  as  it  provides  a  basis  for  review 
and  comparison  in  the  source  selection  process.  Owner¬ 
ship  considerations  cannot  be  disregarded  in  cuch  a 
process . 

10.  There  is  a  need  to  establish  a  uniform  methodology 
for  estimating  personnel-related  cost  factors  as¬ 
sociated  with  life-cycle  cost  projections. 

11.  No  one  is  looking  into  the  fatigue  factor  that  goes 
along  with  combat.  How  much  can  an  operator  or 
maintainer  take  before  giving  up?  This  needs  to  be 
considered  in  system  design. 

12.  Military  personnel  should  be  assigned  to  contractor 
facilities  to  help  familiarize  themselves  with  weapon 
systems  being  procured.  This  could  greatly  reduce 
the  number  of  human-induced  failure  in  systems. 

13.  The  wrong  people  are  designing  manuals  and  training 
courses.  Engineers  write  the  manuals  used  by  high 
school  students . 

14.  The  Government  is  going  to  encounter  serious  dif¬ 
ficulties  in  devising  and  initiating  an  objective, 
meaningful,  timely,  and  equitable  basis  for  reporting 
contractor  quality  histories. 

15.  There  is  a  need  for  well  written  and  definitive 
manufacturing  instructions  which  are  understandable 
at  the  worker  level.  This  is  extremely  important  and 
is  not  always  the  case.  The  effect  of  human  factors 

in  the  day-to-day  production  cycle  in  the  manufacturing 
processes  is  vital.  Units  that  are  difficult  to  as¬ 
semble,  test,  rework,  reassemble,  and  retest  encounter 
many  problems. 

16.  Direct  design  evaluation  for  quality  assurance  by  pro¬ 
gram  offices  in  DOD  is  not  cost  effective.  The  program 
office  should  be  more  concerned  about  evaluating  the 
contractor's  quality  assurance  program  rather  than  the 
system  design. 

17.  A  committee  needs  to  be  established  to  redefine  the 
roles  of  the  various  disciplines  which  make  up 
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systems  engineering.  This  is  necessary  to  accommodate 
the  new  emphasis  on  supportability . 

18.  Industry  must  be  made  more  aware  of  the  problems  the 
military  is  experiencing  with  fielded  systems.  The 
problems  must  be  defined  and  engineering  analysis  done 
to  identify  the  causes.  The  current  military  data 
reporting  systems  are  inadequate  for  this  purpose. 

Contractors  need  to  better  understand  the  emphasis  to 
be  placed  on  logistic  support  in  new  development  pro- 
greuns.  There  is  a  need  to  establish  a  rating  system  for 
use  in  requests  for  proposals  to  convey  this  emphasis  to 
contractors . 
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EXAMPLES  OF  MILITARY  EQUIPMENT  REPORTED  BY 
THE  SERVICES  TO  BE  UNDEPENDABLE  AND  DIFFICULT 


TO  SUPPORT  AND  OPERATE 


System/ equipment 


Problem 


TOW  (Antitank  missile 
system,  ground  version) 


Dragon  (antitank  missile 
system) 


T142  tank  track  on  M60 
series  tank 


AH-1  "Cobra"  attack 
helicopter 


GOER  (transport/resupply 
vehicle) 


MllO  self-propeller 
howitzer  (8  inch) 


Test  and  diagnostic  equip¬ 
ment  used  for  avionics 
and  electrical  sub¬ 
systems 


Battery  power  supplies  were  un¬ 
reliable.  As  a  result,  missile 
launches  were  jeopardized  or 
guidance  was  lost  during  flight. 

Component  malfunctions  plus 
human  factor  problems  cause  many 
of  these  missiles  to  miss  the 
target. 

This  track  must  be  replaced  at 
1,500  to  2,000  miles.  It  is 
less  reliable  than  its  prede¬ 
cessor. 

The  main  rotor  hub  has  signifi¬ 
cant  reliability  problems  due  to 
frequent  failure  of  feathering 
axis  bearings. 

Extreme  bounce  generated  by  vehi¬ 
cle  produced  serious  driver  fa¬ 
tigue.  Numerous  components  suf¬ 
fer  high  rates  of  failure. 

Numerous  hydraulic  components 
problems  being  experienced  since 
recent  modifications  added  a 
heavier  gun  tube.  Additional 
problems  exist  with  road  wheels, 
overheating  engines,  gun  sight¬ 
ing  equipment,  and  projectile 
ramming  systems . 

Equipment  is  unreliable,  requires 
extensive  calibration,  and  is 
difficult  to  repair. 
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System/ equipment 


Navy 


MK-86  gun  fire  control 
system  on  many  surface 
warships 


AN/SPG-55B  guided  missile 
control  radar  on  many 
surface  warships 


AN/sps-40  air  search  radar 
on  many  surface  ships 


Wasteheat  boilers  on 

DD-963  class  destroyers 


BQQ-5  sonar  on  SSN-688 
class  attack  aubmarines 


MK-18  periscope  on  SSN-688 
class  submarines 


S3A  antisulanarine  warfare 
aircraft 


Problem 


A  significantly  large  number  of 
random  failures  eunong  the  40,000 
plus  parts  and  the  inability  of 
supply  system  to  meet  these 
replacement  component  demands 
have  caused  low  operational 
availability. 

Low  reliability,  replacement 
part  shortages,  and  inadequate 
operator  and  maintenance  training 
are  affecting  operational  availa¬ 
bility. 

High  failure  rates  of  some  parts, 
long  time  to  receive  replacement 
parts,  and  inadequate  number  of 
trained  technicians  lead  to 
operational  availability  prob¬ 
lems  . 

Extremely  difficult,  if  not  im¬ 
possible  to  adequately  maintain. 
Equipment  failure  would  result 
in  partial  loss  of  ship' s  elec¬ 
trical  power,  potentially  affect¬ 
ing  ship's  weapon  systems. 

severe  replacement  part  shortages 
have  caused  submarines  to  experi¬ 
ence  mission  degradation. 

Fleet  has  experienced  many  prob¬ 
lems  including  (1)  slip  ring  fail¬ 
ures,  (2)  poor  logistic  support, 
(3)  inadequate  technical  docu¬ 
mentation,  (4)  inadequate  main¬ 
tenance  training,  and  (5)  insuf¬ 
ficient  technical  support  equip¬ 
ment. 

Low  reliability  of  many  key 
electronic  components  have 
caused  low  aircraft  operational 
availability  rates. 
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System/ equipment 
Air  Force 

"Turkey  Feathers"  on  F-15 
aircraft 


F-lOO  engine  in  F-15 
aircraft 


Automatic  test  equipment 
for  F-15  aircraft 


Stability  augmentation 
system  in  A-10  aircraft 


Flight  controls  in  A-10 
aircraft 


Shelters  for  A-10  aircraft 


War  reserve  spare  kits/ 
base  level  self- 
sufficiency  kits 


(951516) 
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Problem 


These  engine  parts  are  wearing 
out  after  about  15  hours  of  use. 
They  cost  $1,000  each,  and  each 
aircraft  has  about  30  of  them. 

Problems  with  reliability  and 
durability,  particularly  in  the 
"hot  section"  of  the  engine, 
have  led  to  low  operational 
availability  rates. 

Problems  include  (1)  lack  of 
adequately  trained  and  experi¬ 
enced  operators  and  maintenance 
personnel,  (2)  some  software 
incompatability ,  and  (3)  low  re 
liability  of  the  built-in  test 
and  avionics  intermediate  shop 
automatic  stations.  These  prob¬ 
lems  degrade  testing  efficiency 
and  ultimately  degrade  aircraft' 
operational  readiness. 

Problems  with  targeting  on  the 
first  201  aircraft  and  with 
vibrations  and  signal  inter¬ 
ruptions  on  the  last  158  air¬ 
craft  affect  the  aircraft's 
mission  effectiveness. 

Clearance  for  the  aircraft 
cables  and  controls  is  not 
sufficient,  and  foreign  objects 
may  jaun  the  controls.  This  con¬ 
dition  may  already  have  contri¬ 
buted  to  aircraft  accidents. 

Serious  shortage  of  shelters 
in  Europe  might  adversely 
affect  maintenance  of  aircraft. 

Shortages  of  war  reserve  re- 
replaconent  parts  and  components 
exist.  These  kits  are  needed 
to  keep  aircraft  and  their 
subsystems  operational . 
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